Health information service system

ABSTRACT

A health information system according to embodiments is a system including a measurement apparatus configured to measure a health and beauty status of a user, an action status acquirement apparatus configured to acquire a history of actions regarding health and beauty performed by the user, and a management apparatus connected over a network to the measurement apparatus and the action status acquirement apparatus. The management apparatus associates health and beauty measurement information with the history of actions and stores the health and beauty measurement information and the history of actions of the user daily in time sequence. The management apparatus extracts a time-series pattern representing a change or no change of the health and beauty status of the user based on the time-series health and beauty measurement information in the past, extracts a health and beauty action performed by the user or a health and beauty action not performed by the user when the extracted time-series pattern is found, and uses the health and beauty measurement information before and after the health and beauty action or in a nonperformance period to produce performance or nonperformance effect evaluation information of the extracted health and beauty action.

TECHNICAL FIELD

Embodiments of the present invention relate to a health informationservice technology regarding health and beauty.

BACKGROUND ART

Mirror apparatuses are conventionally known in which information isdisplayed on a mirror reflecting a subject. Patent Document 1 hasdisclosed a mirror apparatus which can be installed in various types offacilities such as hair salons involving the use of mirrors to display avideo received from a server apparatus and the appearance (mirror image)of a customer (subject) to enable comparison between them.

PRIOR ART DOCUMENT Patent Document

[Patent Document 1] Japanese Patent Laid-Open No. 2004-357099

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention

It is an object of the present invention to assist in improvement ofhealth and beauty of a user based on collected health and beautystatuses of the user and statuses of actions regarding health and beautyperformed by the user.

Means for Solving the Problems

A health information service system according to embodiments includes ameasurement apparatus configured to measure a health and beauty statusof a user, an action status acquirement apparatus configured to acquirea history of actions regarding health and beauty performed by the user,and a management apparatus connected over a network to the measurementapparatus and the action status acquirement apparatus. The managementapparatus includes a storage section configured to associate health andbeauty measurement information about health and beauty of the useracquired by the measurement apparatus with the history of actions and tostore the health and beauty measurement information and the history ofactions of the user daily in time sequence, a time-series data analysissection configured to extract a time-series pattern representing achange of the health and beauty status of the user or a time-seriespattern representing no change of the health and beauty status of theuser based on the time-series health and beauty measurement informationin the past, and a performance effect evaluation section configured toextract the health and beauty action performed by the user when thetime-series pattern is found before and after the health and beautyaction of the user based on the history of actions or the health andbeauty action not performed by the user when the time-series pattern isfound in a nonperformance period of the health and beauty action of theuser based on the history of actions and to use the health and beautymeasurement information before and after the health and beauty action orthe health and beauty measurement information in the nonperformanceperiod to produce performance or nonperformance effect evaluationinformation about the extracted health and beauty action.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 A diagram showing the system configuration and functional blocksof apparatuses of a health information service system according to anembodiment.

FIG. 2A A diagram for illustrating how to use a mirror apparatusaccording to the embodiment.

FIG. 2B A diagram for illustrating how to use the mirror apparatusaccording to the embodiment.

FIG. 3 A diagram for showing an example of display of a health andbeauty action content according to the embodiment.

FIG. 4 A diagram showing functional blocks of a mobile communicationapparatus according to the embodiment.

FIG. 5 A diagram showing an example of user terminal managementinformation according to the embodiment.

FIG. 6 A diagram showing an example of health management informationaccording to the embodiment, and showing exemplary data provided byclassifying information including health and beauty environmentinformation, external service content information, health and beautyaction information, and personal preference evaluation information intime sequence.

FIG. 7 A diagram showing an example of health and beauty predictioninformation according to the embodiment, and showing exemplary dataprovided by classifying information including health and beautyenvironment information, external service content information, andpersonal preference evaluation information in time sequence.

FIG. 8 A diagram showing an example of health and beauty planninginformation according to the embodiment, and showing exemplary dataprovided by classifying information including health and beautyenvironment information, personal preference evaluation information, andhealth and beauty content planning information in time sequence.

FIG. 9 A diagram showing a processing flow in the health informationservice system according to the embodiment.

FIG. 10 A diagram showing a processing flow in the health informationservice system according to the embodiment.

FIG. 11 A diagram showing a processing flow in the health informationservice system according to the embodiment.

FIG. 12 A diagram showing a processing flow of acquiring an actionstatus of a user regarding health and beauty in the health informationservice system according to the embodiment.

FIG. 13A A flow chart showing processing of providing recommendationinformation in accordance with location information of a mobilecommunication apparatus in the health information service systemaccording to the embodiment.

FIG. 13B A flow chart showing processing of providing recommendationinformation in accordance with location information of the mobilecommunication apparatus in the health information service systemaccording to the embodiment.

FIG. 14 A flow chart showing processing of associating locationinformation of the mobile communication apparatus with a history ofactions and processing of providing recommendation information based onthe location information of the history of actions in the healthinformation service system according to the embodiment.

MODE FOR CARRYING OUT THE INVENTION

A preferred embodiment of the present invention will hereinafter bedescribed with reference to the accompanying drawings.

A health information service system according to the embodiment is acomputer system which assists in improvement of health and beauty. Asshown in FIG. 1, the system is configured to include a mirror apparatus100, a control apparatus 200 for the mirror apparatus 100, and a serviceserver apparatus 300 connected to the control apparatus 200 over anetwork. The service server apparatus 300 is also connected to anexternal service server apparatus 400 and a mobile communicationapparatus 500 over networks.

The mirror apparatus 100 and the control apparatus 200 are installed,for example, in a house where a user lives, or in a facility or a shopwhich the user visits. The mirror apparatus 100 functions as a mirrorreflecting a mirror image of the user corresponding to a subject and asa display apparatus for displaying various types of information. Asdescribed later, the mirror apparatus 100 also functions as an apparatusfor acquiring health and beauty statuses and action statuses of the userpresent in front of the mirror apparatus 100.

As shown in FIG. 1 and FIG. 2A, the mirror apparatus 100 includes a halfmirror 111 and a display panel 112. The half mirror 111 is a mirrorhaving a reflecting film formed thereon which reflects a part ofincident light and transmits another part of the incident light, and canbe provided, for example by forming a thin reflecting film onto atransparent substrate made of glass or the like. The rate between thereflected light and the transmitted light can be adjusted by thethickness of the reflecting film. As shown in FIG. 2B, the half mirror111 reflects images of the user incident on the half mirror 111 from thefront side and transmits display light incident from the display panel112 located on the rear side of the half mirror 111 toward the frontside of the half mirror 111.

The display panel 112 is a display apparatus of a self-emitting typewhich has a display screen 113. Examples of the display panel 112include liquid crystal display apparatuses of a back-light type or areflective type, organic EL display apparatuses, and plasma displayapparatuses. The display screen 113 of the display panel 112 can beprovided over the entire half mirror 111 or in a single or a pluralityof particular areas of the half mirror 111.

The mirror apparatus 100 can have any configuration, and for example,the display panel 112 may be a transparent display instead of thedisplay apparatus of the self-emitting type. In this case, thetransparent display apparatus may be formed of an organic EL device oran OLED (Organic Light-Emitting Diode), for example. The organic ELdevice or the OLED typically has a structure in which an organicmaterial is sandwiched between an anode and a cathode each placed on atransparent substrate made of glass or plastic such that the organiclight-emitting material emits light when an electric current is passedtherethrough. Such a device has a characteristic of acting as atransparent film similar to glass during non-display periods with noelectric current passed. When the display panel 112 is formed of thetransparent display apparatus, a mirror which reflects all the ofincident light may be placed on the rear side of the transparent displayapparatus to provide the mirror apparatus 100 configured from thecombination of that mirror and the transparent display apparatus withoutusing the half mirror 111 described above.

The mirror apparatus 100 includes various devices such as a camera 120for taking images of the user, a microphone 130, a speaker 140, a humandetection sensor 150, and sensor device 160. Those devices arecontrolled by the control apparatus 200.

The camera 120 is provided for the mirror apparatus 100 with its imagingrange covering the front side of the mirror apparatus 100. The camera120 can take whole-body images of the user present in front of themirror apparatus 100 or take images of a particular region such as theuser's face.

The camera 120 can be provided by using a visible camera, athermographic camera (far-infrared camera), a near-infrared camera orthe like. For example, as shown in FIG. 1, two cameras 120 can beprovided for the mirror apparatus 100, in which one camera 120 of themis a visible camera and the other camera 120 is a thermographic camera.When one camera can be switched between visible, far-infrared,near-infrared imaging and the like, only one camera 120 may be used.Alternatively, three or more camera 120 may be provided.

The camera 120 may be independent of the mirror apparatus 100 andmounted on the user's back, the ceiling, or a wall of the house totransmit videos taken from the back, the side, or above the user to thecontrol apparatus 200 over a wired LAN or wireless communication.Alternatively, a hairdresser or a massager may use a wearable camera (asmall camera put on his/her body to allow hands-free imaging) totransmit videos taken from the viewpoint of the service provider in realtime over wireless communication or to transmit videos recorded on thecamera over a wired connection or wireless communication to the controlapparatus 200.

For example, it is possible to measure, from images of the user taken bythe near-infrared camera, skin conditions including a skin moisturecontent, blood circulation, spots, makeup staying on or not, makeupdeterioration, and scalp condition. It is also possible to measure abody surface temperature from thermographic images of the user taken bythe far-infrared camera.

Images of the user taken by the visible camera are used to measurevarious types of health and beauty information of the user. For example,parts of the face such as user's eyes, nose, and mouth can be extractedfrom images of the face to measure distortion, sag, or swelling of theface based on coordinates, inclination, and changes over time of therespective regions. In addition, from the measured distortion or thelike of the face, user's facial expression can be analyzed to measurethe presence or absence or the level of a stressful condition. Inaddition to the face images, for example, parts of legs such as a calvesand ankles can be extracted from images of the legs to measure swellingof the legs based on coordinates, inclination of the respective regions,and comparison with numerical values at usual times or from previousmeasurements.

From images of the whole body or images or videos of the upper body ofthe user taken by the visible camera, the body shape of the user can bemeasured. The measured body shape of the user can be matched withpredetermined reference body shape data to calculate distortion of thebody of the user. The camera 120 can be provided by using a monocularcamera, a compound-eye camera, or a multiple camera as appropriate fortaking images of the user necessary for the measurement.

The human detection sensor 150 senses the presence or absence of theuser whose mirror image can be reflected in the mirror apparatus 100.The human detection sensor 150 senses movements of a person in an areaon the front side of the mirror apparatus 100 with infrared rays,ultrasound, visible light or the like to detect whether the user ispresent in front of the mirror apparatus 100. The result of sensing theuser in the human detection sensor 150 is utilized as a trigger forstarting acquisition of a user action status by the mirror apparatus 100or for starting information display to the user. The human detectionsensor 150 can detect whether or not the user has moved out of the frontarea of the mirror apparatus 100, and the detection result can be usedto control, for example, start and stop of functions.

The sensor device 160 includes a plurality of sensor devices such as athermo-hygrometer, an illumination sensor, and a dust sensor formeasuring dirt in the air. The plurality of sensor devices 160 can beconcentrated at a particular point or placed at different points in themirror apparatus 100 as appropriate.

The sensor device 160 can be configured to include the microphone 130.For example, the microphone 130 can pick up the sound of a sneeze or acough of the user, and the number of sneezes can be acquired asinformation about user's health. The sensor device 160 can be configuredto include a motion sensor for detecting motions of the user as well.The motion sensor may be provided by combining the infrared camera ofthe camera 120 with an infrared light-emitting diode.

A health and beauty measurement apparatus 170 is configuredindependently of the mirror apparatus 100 to be a device such as athermometer, a weight scale, a body composition meter, a manometer, askin diagnosis device, a breath odor meter, and a sleep meter. Thehealth and beauty measurement apparatus 170 is connected to the controlapparatus 200 and outputs detection result to the control apparatus 200.

As described above, the health information service system according tothe present embodiment can acquire health and beauty environmentinformation including health and beauty information of the user andenvironment information by using the camera 120, the sensor device 160,and the health and beauty measurement apparatus 170. The camera 120, themicrophone 130, the human detection sensor 150, and the sensor device160 function as a health and beauty environment sensor section of themirror apparatus 100.

The combination of the sensor devices for measuring health and beautystatuses of the user with the sensor devices for measuring environmentinformation is arbitrarily made. In addition, the configuration of thesensor devices in the mirror apparatus 100 is arbitrarily performed, forexample such that the breath odor meter (breath odor sensor formeasuring the breath odor of the user) or the skin diagnosis device isnot provided as the health and beauty measurement apparatus 170independent of the mirror apparatus 100 but is provided in the sensordevice 160 integral with the mirror apparatus 100.

Next, description is made of the control of the mirror apparatus 100 andthe functions of sections of the control apparatus 200. The controlapparatus 200 is configured to include a communication section 211, anoutput control section 212, a storage section 213, a control section220, a sensor control section 231, a measurement data analysis section232, a user action recognition section 233, a user operation recognitionsection 234, an AR control section 235, and a user preferenceacquisition section 236.

The communication section 211 is a communication control section forcommunicating with the mirror apparatus 100 and with the health andbeauty measurement apparatus 170 and performing communication betweenthe control apparatus 200 and the service server apparatus 300 over thenetwork. The communication between the control apparatus 200 and thehealth and beauty measurement apparatus 170 can be performed throughproximity wireless communication such as Bluetooth® or NFC (Near FieldCommunication). Alternatively, the health and beauty measurementapparatus 170 and the control apparatus 200 may have a wired connectionbetween them to perform communication.

The control apparatus 200 and the service server apparatus 300 areconnected to each other over the network such as a LAN (Local AreaNetwork) or the Internet whether it is a wired or wireless channel or adedicated channel.

The output control section 212 performs display control for controllingdrive of the display panel 112 to display various types of informationon the display screen 113 (information output) and sound reproductioncontrol for outputting predetermined audio data through the speaker 140(audio output).

The storage section 213 stores user information registered with themirror apparatus 100, theme content information associated with theuser, mirror setting information, local cache information, and privacycontent information.

The user information includes a user ID of a registered user, name, age,nickname, preference, biometric authentication information forconfirming the identity of the user, date and time of registration, dateand time of service use, frequency of service use, and avatarinformation. The theme content information includes screen displaycontent data such as a theme (wallpaper image) and an icon on thedisplay screen 113 and audio content data such as BGM and sound effects.

The mirror setting information includes device identifying informationof the mirror apparatus 100, manufacture information, networkinformation, connectable device information, and required/not-requiredsetting information for privacy processing of the health and beautyenvironment information acquired from the health and beauty environmentsensor section and/or health and beauty measurement apparatus 170.

The local cache information includes a history of the final screendisplay content and audio content used last by the user, health andbeauty environment information of the user measured by the health andbeauty environment sensor section and the health and beauty measurementapparatus 170, and external service content information acquired fromthe external service server apparatus 400 via the service serverapparatus 300. Examples of the external service content informationinclude information regarding health and beauty such as meteorologicalinformation including weather, temperature and humidity, atmosphericpressure, and rain cloud information, health forecast informationassociated with the meteorological information about ultraviolet rays,heatstroke, rheumatism, depressed mood, asthma, and pollen allergy,traffic information, event information, and commodity information.

The external service content information can include health and beautyaction contents. The health and beauty action contents are images orvideos (action support display data) for supporting user actionsregarding health and beauty such as massage, stretching, gymnastics, andexercise.

The local cache information also includes personal preference evaluationinformation of the user acquired by the user preference acquisitionsection 236 described later.

The privacy content information is sensitive content informationconcerning user's privacy which the user does not want to transmit tothe service server apparatus 300 such as images or videos taken by thevisible camera used to measure health and beauty statuses.

In the present embodiment, the health and beauty environment informationof the user measured by the health and beauty environment sensor sectionand the health and beauty measurement apparatus 170 is transmitted tothe service server apparatus 300. However, the measured original datasuch as images or videos is not transmitted as it is to the serviceserver apparatus 300, but is subjected to privacy processing, forexample by converting the data into another data form, and theinformation after the privacy processing is transmitted as the healthand beauty environment information of the user to the service serverapparatus 300.

The privacy processing is performed by the measurement data analysissection 232 and involves, for example, quantifying information aboutuser's health and beauty obtained from videos or images taken by thevisible camera. Specifically, as described above, the privacy processingfor a skin moisturizing level and skin conditions of the user isperformed by first analyzing the skin conditions of the user including askin moisture content, blood circulation, spots, makeup staying on ornot, makeup deterioration, and scalp condition. Then, the analysisresult is quantified as a skin moisturizing level or a skin conditionscore. For example, the analysis result can be quantified with respectto a predefined evaluation standard or evaluation score to calculate theskin moisturizing level or the skin condition score from the skinmoisture content, blood circulation and the like obtained from theimages of the user.

The privacy processing for whole-body thermographic informationinvolves, for example, setting classification patterns of thermographicinformation in advance and finding and outputting a matchingclassification pattern as the thermographic information of the user.Examples of the classification patterns set in advance can include apattern representing a low body surface temperature of fingers and toesand a pattern representing a low body surface temperature mainly of alower body.

The health and beauty statuses of the user such as distortion of thewhole body or body regions, sag, and swelling can be quantified. Forexample, parts of the face such as user's eyes, nose, and mouth can beextracted from images of the face to measure coordinates and inclinationof those regions to analyze differences from reference body shape dataor changes of those regions over time. Then, the analysis result isquantified by using a predefined evaluation value for distortion amountor displacement amount, and thus the degrees of face distortion, sag,and swelling can be calculated in accordance with the analysis result.In addition, the degree of facial expression (feeling) can also beevaluated from the images of the face. For example, classificationpatterns of images of the face representing feelings such as cheerful,stern, somber, and tight are set in advance and are matched with imagesof the face to output a stress condition of the user in eachclassification pattern.

Only the information quantified through the privacy processing in thismanner can be transmitted to the service server apparatus 300, and theoriginal videos or images can be stored as the privacy contentinformation in the storage section 213. The privacy content informationcan be configured to be protected in the control apparatus 200 by apredetermined security function used as a countermeasure againstinformation leakage such as encryption processing or storing in aspecial storage area provided with a tamper resistance capability.

The control section 220 is responsible for controlling the overallcontrol apparatus 200, and specifically, controls the functions of thecommunication section 211 and the output control section 212 andcontrols input and output of data to and from the storage section 213.

The control section 220 also controls the functions of the sensorcontrol section 231, the measurement data analysis section 232, the useraction recognition section 233, the user operation recognition section234, the AR control section 235, and the user preference acquisitionsection 236 in order to acquire health and beauty statuses and actionstatuses of the user present in front of the mirror apparatus 100.

The sensor control section 231 controls the operations of the camera120, the human detection sensor 150, and the sensor device 160constituting the health and beauty environment sensor section, andacquires, from those devices, images and videos taken and detectionvalues detected by those sensors.

The measurement data analysis section 232 produces health and beautyenvironment information of the user from the result of measurementperformed by the health and beauty environment sensor section and thehealth and beauty measurement apparatus 170. Specifically, as describedin the above privacy processing, the measurement data analysis section232 analyzes skin conditions of the user including a skin moisturecontent, blood circulation, spots, makeup staying on or not, makeupdeterioration, and scalp condition based on near-infrared images orvideos of the user taken by the camera 120. The measurement dataanalysis section 232 also extracts parts of the face such as user'seyes, nose, and mouth from visible images taken by the camera 120 tomeasure coordinates and inclination of those regions to analyzedifferences from the reference body shape data or changes of thoseregions over time.

The measurement data analysis section 232 can perform the privacyprocessing as described above by extracting the classification patternmatching the thermographic information of the user or extracting theclassification pattern matching the user's facial expression (stresscondition). The measurement data analysis section 232 also performs theabove quantification processing on the analysis result.

The user action recognition section 233 calculates location informationof each region of the whole body or the upper body of the user presentin front of the mirror apparatus 100 two-dimensionally orthree-dimensionally from images or videos of the user acquired by thecamera 120 to recognize an action performed by the user in front of themirror apparatus 100.

For example, the user action recognition section 233 refers to healthand beauty action contents for guiding user actions such as stretching,exercise, lower body massage, and face massage to find whether the userhas performed any action or not from the images or videos of the usertaken by the camera 120. When the user has performed any action underthe guidance of a health and beauty action content, the user actionrecognition section 233 can recognize, for example, that the user hasmassaged his/her face.

As shown in FIG. 3, the output control section 212 can acquire thehealth and beauty action contents in advance as the external servicecontent information from the service server apparatus 300, and istriggered by a user operation to display any health and beauty actioncontent on the display screen 113 of the display panel 112. The user cancause his/her mirror image to be superimposed on the displayed data ofthe health and beauty action content in the half mirror 111 and performthe action in synchronization with the video of the health and beautyaction content.

The user action recognition section 233 calculates the locationinformation of each region such as user's face, arm and leg, and torsoin images or videos taken by the camera 120 or acquired by the motionsensor and matches the location information with the correspondingpositions in the displayed data of the health and beauty action contentdisplayed on the display screen 113. From the matching result, the useraction recognition section 233 knows the user action performed inaccordance with changes of the displayed data of the health and beautyaction content. The health and beauty action content can includeinformation for showing an utterance guide which prompts the user to letout his/her voice. The user action recognition section 233 can acquireutterance data of the user picked up by the microphone 130 to performmatching of the utterance timing as well as the matching of each bodyregion.

For example, the user action recognition section 233 can express how thereproduced health and beauty action content matches the positions ofmoving arms and legs or the utterance rhythms in numerical formrepresenting a matching rate. The matching rate can be displayed on thedisplay screen 113 as the health and beauty action result after thecompletion of the user action such as exercise performed in accordancewith the health and beauty action content or can be displayed aplurality of times at predetermined time intervals after the start ofreproduction of the health and beauty action content or after thecompletion of part of the health and beauty action during thereproduction of the health and beauty action content.

The user action recognition section 233 can store the matching rate asan action performance history of the user. For example, the matchingrate can be used to express the degree of completion of the health andbeauty action content in numerical form to evaluate the actionachievement of the user.

The user action recognition section 233 may use the matching rate torecognize that the user has performed the action of the health andbeauty action content (such as exercise), for example when the matchingrate is higher than a predetermined threshold value. Alternatively,regardless of the matching rate, the user action recognition section 233may determine that the action achievement of the user in accordance withthe health and beauty action content is made when the camera 120 or thehuman detection sensor 150 senses that the user is present in front ofthe mirror apparatus 100 for a certain time period during which thedisplayed data of the health and beauty content is displayed on thedisplay screen 113.

In contrast, when the matching rate is lower than the predeterminedthreshold value, the user action recognition section 233 may use thematching rate to recognize that the user has not performed the action ofthe health and beauty action content. Alternatively, regardless of thematching rate, the user action recognition section 233 may determinethat any action achievement of the user in accordance with the healthand beauty action content is not made when the camera 120 or the humandetection sensor 150 senses that the user is absent in front of themirror apparatus 100 for a certain time period during which thedisplayed data of the health and beauty content is displayed on thedisplay screen 113.

The user action recognition section 233 stores a history of user actionsincluding the health and beauty action content performed by the user,health and beauty action content performed/not-performed, performancestart time, performance end time, number of performances, andperformance score (matching rate) in the storage section 213. The actionhistory is transmitted to the service server apparatus 300 through thecommunication section 211 at an arbitrary time to store health andbeauty action information in a storage section of the service serverapparatus 300.

The health and beauty actions performed by the user can be configured toinclude actions regarding health and beauty other than exercise, such astaking supplements and listening to healing BGM. A history of supplementtaking can be obtained, for example such that user's intention of makingoperation to indicate whether supplements are taken or not is recognizedby the user operation recognition section 234, later described, and theaction of taking supplements can be acquired as an action regardinghealth and beauty. In addition, a user action regarding health andbeauty of listening to healing BGM can be acquired, for example bydetermining whether or not the user has reproduced music data stored inthe storage section 213 or acquiring a reproduction time period.

The user operation recognition section 234 recognizes and convertsuser's intention of making operation with voice, gestures with arms,hands, fingers, or legs, sigh, sightline, and face direction into apredetermined operation signal and outputs the converted operationsignal to the control section 220 to allow the control section 220 tocontrol the operations of the components. For example, predeterminedsounds are associated with operation signals in advance, and user'svoice is picked up by the microphone. The picked up voice can be matchedwith the operation signals to extract a matching operation signal. Inanother example, a predetermined gesture, sightline, and face directionare associated with operation signals, and an image of the user imagetaken by the camera 120 is matched with the operation signals to extracta matching operation signal. In this manner, the user operation can berecognized by the user operation recognition section 234, and inaccordance with the recognized user operation, control can be performedsuch as display of a particular screen on the display screen 113 of themirror apparatus 100, control of reproduction of a content, and turn-upand turn-down of the volume.

The user operation recognition section 234 can recognize produced soundsor movements which do not show user's obvious intention of makingoperation such as yarn, sigh, stomach rumbling, gas, and facialexpression. In this case, the user operation recognition section 234 canperform control, such that the condition of the user is found from thatrecognized information to perform screen display or audio output asappropriate for the found condition of the user. For example, when thesound of a sneeze is recognized, the user operation recognition section234 may add effects by showing a display screen which represents wavescausing ripples on the display screen 113 associated with the sneeze ormay display a character on the display screen 113 which speaks to theuser with a reproducing sound, “are you cold?”

Next, description is made of the AR control section 235. The user actionrecognition section 233 acquires the action status of the user who hasperformed the action in front of the mirror apparatus 100. The ARcontrol section 235 shows the display data in synchronization with usermovements over the image of the user reflected as the mirror image inthe mirror apparatus 100.

The AR control section 235 can perform, for example, augmented realityprocessing of augmenting the reality in which a display content istransparently superimposed from the display panel 112 at a specifiedposition on the mirror image of the user reflected optically in themirror surface. The display content includes display locationinformation indicating at which position of the mirror image the displaydata needs to be superimposed.

First, the user action recognition section 233 uses the result ofmeasurement of the user present in front of the mirror apparatus 100acquired by the health and beauty environment sensor section tocalculate location information of each region of the whole body or theupper body of the user two-dimensionally or three-dimensionally. The ARcontrol section 235 acquires the location information of each region ofthe user from the user action recognition section 233 to calculatedisplay coordinates and a display size of the display content based onthe location information of each region of the user. The AR controlsection 235 outputs the calculated display coordinates, display size,and display content to the output control section 212, and the outputcontrol section 212 displays the display content with the specifieddisplay coordinates and display size on the display screen 113. Thedisplay content for use in the augmented reality processing may includeinformation such as display angle, distortion, color, and brightness.

The user preference acquisition section 236 collects living behaviors, ahealth and beauty feeling representing how the user feels the health andbeauty status, and information about preference in health and beautyactions (user preference evaluation information) from the user. The userpreference information can be acquired through the mirror apparatus 100.

The user preference acquisition section 236 can operate at apredetermined time. For example, the collection of the health and beautyfeeling is performed at the time when the user appears in front of themirror apparatus 100 first in the morning, and at the time when the useris detected by the mirror apparatus 100 after a predetermined timeperiod of absence in front of the mirror apparatus 100 in the evening orlater. The collection of the health and beauty action preference isperformed at the time when the reproduction of a health and beautycontent is completed or at the time when a health and beauty action ofthe user associated with a health and beauty action content iscompleted.

For starting the acquisition of user preference, the user preferenceacquisition section 236 outputs a control signal to the output controlsection 212 so as to display a user preference characteristic collectingcontent for collecting user preference characteristics, that is, a userpreference acquisition screen on the display screen 113. When the userpreference characteristic collecting content for collecting data aboutuser preference is displayed by the output control section 212, the userpreference acquisition section 236 acquires a response to the userpreference characteristic collecting content from the user recognized bythe user operation recognition section 235.

1) For a living activity record, when the mirror apparatus 100 isinstalled in an undressing room, the camera 120 senses that the user istaking off clothes before taking a bath and the user preferenceacquisition section 236 acquires the time of the sensing as a bath time.At this point, the facial expression, body color, and body surfacetemperature of the user at usual times taken previously by the camera120 are compared with the facial expression, body color, and bodysurface temperature of the user taken by the camera 120 before the bath.When the body (face) has a reddish color, the user preferenceacquisition section 236 outputs a sound, “did you drink much today?”from the speaker 140 to ask the user. The user responds to the askingsound by his/her voices expressing yes or no and an amount of alcoholintake if the user drank. The response of the user is picked up by themicrophone 130, the picked up response of sound is analyzed, and theuser preference acquisition section 236 collects data about the presenceor absence of alcohol intake and the amount of alcohol intake, if any.

2) For the health and beauty feeling, the user preference characteristiccollecting content for asking the user about current physical condition,mood, any pain in body parts, and stiff shoulders is displayed on thedisplay screen 113. The user can make answers on the physical condition,mood, any pain in body parts, and stiff shoulders by shaking his/herhead (shaking his/her head up and down for Yes, and shaking his/her headfrom side to side for No), making other gestures with hands, fingers, orbody, or making sounds. These user operations are recognized by the useroperation recognition section 234, and the recognition result is outputto the user preference acquisition section 236 as the response to thequestions about the physical condition, mood, any pain in body parts,and stiff shoulders.

3) For the health and beauty action preference, the user preferenceacquisition section 236 can ask whether the user likes the currentlyperformed health and beauty action content or not, whether it ispleasant or not, and whether the user is interested in it or not, in thesame manner as in (2), and can acquire responses.

The control section 220 stores the user preference evaluationinformation (the result of responses to the questions to the user)acquired by the user preference acquisition section 236 as local cacheinformation in the storage section 213. The user preference evaluationinformation is transmitted to the service server apparatus 300 throughthe communication section 211 at an arbitrary time and is stored in thestorage section of the service server apparatus 300.

The user preference information may also be acquired from the mobilecommunication apparatus 500 carried by the user. Examples of the mobilecommunication apparatus 500 include a portable information terminalhaving a data communication function and a computing function (forexample, CPU) such as a multifunctional cellular phone and PDA (PersonalDigital Assistant), a notebook computer, and a tablet computer.

FIG. 4 is a diagram showing the functional blocks of the mobilecommunication apparatus 500. A display section 510 is a displayapparatus and can include an operation input function of a touch panelor the like. A control section 530 can control display of various typesof information and screens based on user input operations performedthrough the display section 510.

A camera 521 has a configuration similar to that of the camera 120 ofthe mirror apparatus 100 and is a visible camera or a near-infraredcamera. A sensor device 522 has a configuration similar to that of thesensor device 160 of the mirror apparatus 100 and includes a pluralityof sensor devices such as a thermo-hygrometer, an illumination sensor, adust sensor, and an optical coherence tomographic device. The sensordevice 522 may be configured to include a microphone 523.

A communication section 531 is similar to the communication section 211of the control apparatus 200 and is a communication control section forcommunicating with the health and beauty measurement apparatus 170 andcommunicating with the service server apparatus 300 over the network. Anoutput control section 532 performs display control for displayingvarious types of information on the display section 510 (informationoutput) and sound reproduction control for outputting predeterminedaudio data (audio output). A location information acquisition section533 functions as a GPS apparatus for acquiring the current position ofthe mobile communication apparatus 500.

The mobile communication apparatus 500 further includes a sensor controlsection 535 and a measurement data analysis section 536. Since thefunctions of those sections are identical to those of the sensor controlsection 231 and the measurement data analysis section 232 of the controlapparatus 200, respectively, description thereof is omitted. The mobilecommunication apparatus 500 can use the devices such as the camera 521,the sensor device 522, and the microphone 523 to measure and acquireinformation other than the information acquired only by the mirrorapparatus 100 such as body misalignment and thermographic patterns inthe health and beauty information. In contrast to the mirror apparatus100, the mobile communication apparatus 500 can transmit, to the serviceserver apparatus 300, the information measured by the devices such asthe camera 521, the sensor device 522, and the microphone 523 inassociation with location information for determining the location ofthe measurement acquired by the location information acquisition section533 serving as the GPS apparatus. In this case, the measured informationand the location information for determining the location of measurementcan be stored in association with each other in a storage section 314 ofthe service server apparatus 300.

A user action recognition section 537 can be configured, for example, todisplay a health and beauty action content of facial lymphatic massageover a face image of the user taken by the camera 521 on the displaysection 510 to recognize the user action performed in synchronizationwith the video of the health and beauty action content, similarly to theuser action recognition section 233 of the control apparatus 200. Theresult of the recognition of the health and beauty action can be storedin association with location information about the location of theperformance of the health and beauty action acquired by the locationinformation acquisition section 533 serving as the GPS apparatus.

The user action recognition section 537 calculates location informationof each region of use's face, arms and hands in images or videos takenby the camera 521 and matches the location information with thecorresponding positions in the displayed data of the health and beautyaction content displayed on the display section 510. From the matchingresult, the user action recognition section 537 can know the user actionperformed in accordance with changes of the displayed data of the healthand beauty action content.

The mobile communication apparatus 500 includes a user preferenceacquisition section 538 having the function similar to that of the userpreference acquisition section 236 and functioning to collect userpreference information at a predetermined time. For example, the currentposition is found by the location information acquisition section 533,and the time when the user returns home can be acquired as a go-hometime in health and beauty action preference information. In addition,from the location information recorded in association with theperformance of a health and beauty action, the user preferenceacquisition section 538 can recognize the location where the userperforms the health and beauty action (for example, in his/her company,on the move during commute, in a park, and any other facility) and canset that location as information indicating the preference of the userin performing the health and beauty action (the preferred location ofperformance).

The user preference acquisition section 538 can collect, as a get-uptime, data about the time when the user turns off a ringing alarm clockset for morning wake-up in the mobile communication apparatus 500. Inanother example, the mobile communication apparatus 500 can beconfigured to receive the result of morning measurement with thethermometer (health and beauty measurement apparatus 170) and collectdata about the time of the reception of the morning temperaturemeasurement result from the thermometer as the get-up time.

The mobile communication apparatus 500 can be configured to receive theresult of measurement with the sleep meter (health and beautymeasurement apparatus 170) and acquire a fall-asleep time, a wake-uptime, or a get-up time, as the user preference information.

The camera 521 of the mobile communication apparatus 500 can take imagesof breakfast, lunch, and dinner, the calorie intake of each meal can becalculated with a calorie calculation application installed in advanceon the mobile communication apparatus 500, and the resulting informationcan be acquired as the user preference information. The caloriecalculation can also be performed, for example by connecting to a sitewhere a calorie calculation application is provided, transmitting thetaken images of menus thereto from the mobile communication apparatus500, and then receiving the calorie calculation result therefrom.Alternatively, the user may input caloric values to the mobilecommunication apparatus 500 without taking images of menus.

The user preference information acquired by the user preferenceacquisition section 538 of the mobile communication apparatus 500 isstored in a storage section 534 and is transmitted to the service serverapparatus 300 through the communication section 531 for storage as theuser preference evaluation information in the storage section of theservice server apparatus 300. The location information at the time ofthe acquisition of the user preference may also be transmitted to theservice server apparatus 300 and stored as user preference evaluationinformation in association with the location information in the storagesection 314 of the service server apparatus 300.

Next, description is made of the service server apparatus 300 in detail.The service server apparatus 300 is configured to include acommunication section 311, an external content acquisition section 312,an output control section 313, the storage section 314, a controlsection 320, a time-series data analysis section 331, a performanceeffect evaluation section 332, a user preference evaluation section 333,a health and beauty prediction section 334, a recommendation section335, a personal learning and planning section 336, and a collectiveintelligence data analysis section 337.

The communication section 311 is a communication control section forcommunicating with the control apparatus 200 and with the mobilecommunication apparatus 500 over the network. The external contentacquisition section 312 acquires various types of external contentsdescribed above from the external service server 400 over the network.

The storage section 314 stores user information, user terminalmanagement information, health management information, health and beautyprediction information, and health and beauty planning information(personal study planning information). The user information is identicalto the user information stored in the control apparatus 200 of eachmirror apparatus 100 and is personal information of the user describedabove. The user terminal management information is informationindicating the relationship between the user ID and device types anddevice IDs for identifying the mirror apparatus 100 and the mobilecommunication apparatus 500. The user terminal management informationcan be configured to include the device type and the device ID of themobile communication apparatus 500.

FIG. 5 is a diagram showing an example of the user terminal managementinformation. A single user can have a plurality of devices, or two userscan share the same mirror apparatus 100. The mirror apparatus 100 can beconfigured to have a plurality of types with different sizes from amirror (large) for reflecting the user's whole body, a mirror (medium)for reflecting the upper body, to a mirror (small) mountable on a tablefor mainly reflecting the face, different mirror image ranges, differentacquirable health and beauty environment information, differentreproducible health and beauty contents, and different recognizablehealth and beauty actions. For each of the large, medium, and smallmirror apparatuses 100, the size of the mounted display screen 113, thesensor device 160, the camera 120, the speaker 140, and the humandetection sensor 150 are provided as appropriate for the size andconfiguration of the mirror apparatus 100 on which they are mounted. Theuser and the mobile communication apparatus 500 may have a one-to-onerelationship, or a plurality of users may share the mobile communicationapparatus 500.

FIG. 6 is a diagram showing an example of the health managementinformation. The health management information is stored for each userand can be configured as data provided by organizing health and beautyenvironment information, external service content information, healthand beauty action information, and personal preference evaluationinformation in time sequence as shown in FIG. 6.

The health and beauty environment information includes health and beautyinformation such as basal body temperature, menstrual cycle, weight,degree of body misalignment, degree of face sag, degree of faceswelling, skin moisturizing level, skin condition score, thermographicpattern, sleep quality, degree of leg swelling, level of breath odor,facial expression, and number of sneezes, and environment informationincluding indoor temperature and indoor humidity.

Each of the health and beauty information and the environmentinformation is the results of measurement in the devices acquiredthrough the mirror apparatus 100 and the information processed on thebasis of the measurement results in the control apparatus 200. Since theinformation is described above, description thereof is omitted.

The external service content information is information about health andbeauty such as meteorological information including weather, temperatureand humidity, atmospheric pressure, and rain cloud information, healthforecast information associated with the meteorological informationabout ultraviolet rays, heatstroke, rheumatism, depressed mood, asthma,and pollen allergy, traffic information, event information, andcommodity information, and is obtained from the external service serverapparatus 400 of the service system.

The health and beauty action information is information indicating ahistory of performed health and beauty actions for improvement,maintenance, and healing in health and beauty of user's mind and bodysuch as massage, stretching, exercise, and music listening, andcorresponds to a history of user actions performed in accordance withhealth and beauty action contents described above.

The personal preference evaluation information includes a livingactivity record including get-up time, bedtime, times and calorieintakes of breakfast, lunch, and dinner, go-home time and bath time, andamount of alcohol intake, the health and beauty feeling acquired fromthe user including physical condition, mood, headache, backache, stiffshoulders, and condition and degree of joint pain, and health and beautyaction preference including the statuses of performance of health andbeauty actions and user's liking for or the user being good or bad ateach health and beauty action acquired from the user. The health andbeauty action preference can be configured to include preferenceinformation such as taking supplements or listening to healing BGM. Thepersonal preference evaluation information corresponds to the personalpreference information acquired by the user preference acquisitionsections 236 and 538.

The time-series data analysis section 331 analyzes the health and beautyinformation of the user acquired in a time sequence to produce a patternof health and beauty or a pattern of health and beauty status changes(time-series pattern) for the user. The time-series data analysissection 331 can perform the time-series data analysis processing at anarbitrary time once the health information has been accumulated in thestorage section 314 over a certain time period necessary for thetime-series data analysis. The result of the time-series data analysisprocessing is stored in the storage section 314. The time-series dataanalysis processing may be performed by using the result of thepreceding time-series data analysis.

By way of example, assuming that thermographic images are subjected tothe time-series data analysis, patterns of thermographic images taken atsubstantially the same time of day before taking a bath and data ofindoor temperature can be used to see changes in body surfacetemperature over time and a tendency of the changes is analyzed. Fromthe thermographic image, a coldness classification type such as“coldness of fingers and toes” and “coldness of lower body” or acoldness level (high, medium, or low) can be known as described above,so that the relationship between daily changes of the type and factorscan be analyzed.

For example, the relationship between meteorological information such asweather or atmospheric pressure and coldness classification type, or therelationship between sleep quality and coldness classification type isanalyzed in time sequence. The analysis can create the analysis resultin which the cold lower body is found in bad weather and low sleepquality leads to a “high” coldness level.

When an “ovulation day” comes in the menstrual cycle, the backache turnsinto a “high” or “medium.” level. When the dispersing pollen countexceeds 50/m³, the number of sneezes exceeds 20. When the amount ofalcohol intake is “large” and the bedtime is “after 12 o'clock,” theproduced analysis result can be that there are tendencies of the degreeof face swelling to be “5 or higher,” the degree of leg swelling to be“5 or higher,” the get-up time to be “after 8 o'clock,” and the user“not to perform” a health and beauty action scheduled in the morning(for example, calisthenics). When it is “cloudy” or “rainy,” a tendencyof a joint pain to turn into a “medium” or “strong” level can beproduced as the analysis result.

The time-series data analysis section 331 can use health and beautyinformation of the user acquired in time sequence to produce thetime-series pattern of health and beauty status changes of the userdescribed above and to produce a time-series pattern of health andbeauty statuses of the user which have not been changed. For example,from the thermographic images, the time-series data analysis section 331can produce a time-series pattern of health and beauty statuses of theuser which represent no changes in coldness level from “high coldness oflower body” to “high coldness of lower body.”

The performance effect evaluation section 332 analyzes a change beforeand after a health and beauty action to perform performance effectevaluation processing for evaluating the health and beauty actionperformed by the user. For example, the performance effect evaluationsection 332 finds a change in numerical value of health and beautyinformation before and after a health and beauty action, a change incoldness classification type of thermographic images, for example from“coldness of fingers and toes” to “coldness of lower body,” or a changein coldness level, for example from “high coldness of lower body” to“low coldness of lower body.”

The performance effect evaluation section 332 can perform theperformance effect evaluation processing on health information acquiredover a predetermined time period within a time period such as a week ora month during which the health and beauty action is continued, or canperform the performance effect evaluation processing on healthinformation acquired over a time period from the start date of thehealth and beauty action to the present time.

After finding such a change, the performance effect evaluation section332 evaluates the degree of the change. For example, when the coldnessclassification type changes from “high coldness of lower body” to “lowcoldness of lower body,” the performance effect evaluation section 332can make an evaluation to determine that the health and beauty actionperformed by the user had a performance effect and was a factor inchanging the health and beauty status of the user. Similarly, when thedegree of face swelling changes to a lower level or the weight isreduced, the performance effect evaluation section 332 can make anevaluation to determine that a performance effect was created.

The evaluation can be performed by setting an evaluation standard inadvance and determining “performance effect created” when the evaluationstandard is met. In addition, evaluation standards can be finely set toadd performance effect levels such that “performance effect created(small)” is determined when a change in numerical value is 5% and“performance effect created (large)” is determined when a change innumerical value is 10% or higher.

In contrast, when the coldness classification type changes from “lowcoldness of lower body” to “high coldness of lower body,” theperformance effect evaluation section 332 can make an evaluation todetermine that the health and beauty action of the user has noperformance effect. Similarly, when the degree of face swelling changesto a higher level or the weight is increased, the performance effectevaluation section 332 can make an evaluation to determine that noperformance effect is created, and can also add levels of no performanceeffect depending on the degree of the change.

The performance effect evaluation processing may be performed by takingaccount of the history of performed health and beauty actions inconnection with the degree of changes of health and beauty information.For example, the action records of health and beauty actions includingcontinuous performance time, number of performances, and performancescore can be used to evaluate the level (high, medium, or low) ofperformance effect created/performance effect not created. The actionrecords can include a case where the user has not performed the healthand beauty action. In addition, collective intelligence analysisinformation, described later, can be used and compared with averagechanges in numerical value of health and beauty in a group of users ofthe same age to evaluate the level (high, medium, or low) of performanceeffect created/performance effect not created.

The performance effect evaluation section 332 can evaluate not only theperformed health and beauty actions but also the effect provided by notperforming a scheduled health and beauty action. For example, theperformance effect evaluation section 332 evaluates the degree ofinfluence of an unperformed health and beauty action based on changes innumerical value of health and beauty information such as skin conditionscore, weight, and body fat during a time period (nonperformance period)in which exercise or massage scheduled as a health and beauty action isnot performed. Thus, the history of user actions acquired as health andbeauty actions includes the record of nonperformance of the health andbeauty action content, and the user action recognition section 233 canrecognize that the user has not performed the action in accordance withthe scheduled health and beauty action content.

For example, when a health and beauty action content is not reproduced,when a user action performed in accordance with changes of displayeddata of a health and beauty action content cannot be found over apredetermined time period or longer during reproduction of the healthand beauty action content, or when the absence of the user is sensed bythe human detection sensor 150 over a predetermined time period orlonger during reproduction of a health and beauty action content, theuser action recognition section 233 can recognize that the user has notperformed that health and beauty action content.

The performance effect evaluation section 323 extracts a nonperformanceperiod of predetermined days or longer during which the user has notperformed a health and beauty action from the storage section 314 andfounds in the extracted nonperformance period, for example, a patternindicating a change from “low coldness of lower body” to “high coldnessof lower body” extracted by the time-series data analysis section 331.The performance effect evaluation section 323 can extract the health andbeauty action not performed by the user when the pattern of the changeis found during the nonperformance period of the health and beautyaction and can use each health and beauty measurement information in thenonperformance period, for example, each health and beauty measurementinformation at the start and end of the nonperformance period to producenonperformance effect evaluation information of the extracted health andbeauty action. In this manner, the performance effect evaluation section323 can produce the evaluation information from the evaluation of theeffect, change, and outcome caused by the nonperformance of the healthand beauty action such as worsening of coldness from “low coldness oflower body” to “high coldness of lower body” as a result ofnonperformance of the predetermined health and beauty action.

When an obvious change is found in the numerical value, image, orpattern of health and beauty information other than the health andbeauty information on which a performed health and beauty action isassumed to act, that change can be evaluated as a performance effect.For example, health and beauty massage assumed to have a coldnessremoving effect may cause changes in health and beauty status of theuser such as metabolism speed-up, increased skin condition score,improved sleep quality, and increased number of “cheerful” facialexpressions, and those changes are evaluated as a secondary health andbeauty action effect.

The performance effect evaluation section 332 of the present embodimentcan evaluate the performance effect of a health and beauty action basedon a pattern representing no change of the health and beauty status ofthe user as well as a pattern representing a change of the health andbeauty status of the user. For example, in a time-series pattern of thehealth and beauty status of the user representing no change in coldnesssuch as from “high coldness of lower body” to “high coldness of lowerbody,” the performance effect evaluation section 332 extracts a healthand beauty action producing no change in health and beauty status or ahealth and beauty action not performed by the user during thenonperformance period of the health and beauty action. The performanceeffect evaluation section 332 can use each health and beauty measurementinformation before and after the health and beauty action or each healthand beauty measurement information during the nonperformance period toproduce nonperformance effect evaluation information of the extractedhealth and beauty action. Since “high coldness of lower body” does notchange, the performance effect evaluation section 332 can make anevaluation to determine that a predetermined health and beauty actionperformed by the user has no performance effect or that nonperformanceof a predetermined health and beauty action is a factor in the unchangedhealth and beauty status.

The user preference evaluation section 333 performs user preferenceevaluation processing of evaluating personal preference of the userbased on each information such as the living activity record, actionplanning, health and beauty feeling, and health and beauty actionpreference of the user at an arbitrary time.

For example, the user preference evaluation section 333 can estimate alife rhythm pattern such as “active in morning,” “active in night,” or“irregular,” based on a gate-up time and a bedtime within apredetermined time period in the living activity record. Theclassification patterns “active in morning,” “active in night,” and“irregular” can be associated with predetermined get-up times andbedtimes in advance, and the user preference evaluation section 333 canrefer to average values of get-up times and bedtimes of the user in thepredetermined time period to determine which classification pattern theymatch, thereby estimating the life rhythm pattern of the user.

In addition, the user preference evaluation section 333 can know thetendency of the user to get up after 9 o'clock and not to perform ahealth and beauty action in the next morning of the alcohol intake of a“medium” or “large” amount. In this case, the user preference evaluationsection 333 can extract, for example, time-series data associated withthe “medium” or “large” amount of alcohol intake and extract a get-uptime and a history of health and beauty actions from time-series data ofthe day after the date of the extracted time-series data to find useractivity situations such as the get-up time and health and beauty actionon the next day in relation to the amount of alcohol intake.

Based on the health and beauty action preference, the tendency of theuser to be good or bad at can be found for each of a plurality of healthand beauty actions. For example, the user preference evaluation section333 can use the health and beauty action preference and the actionhistory of the health and beauty action information to find thatmassage, stretching, and calisthenics are “pleasant” and “performedevery day” when the user performs those health and beauty actions apredetermined number of times or more and feels “pleasant” as the actionpreference. The user preference evaluation section 333 can alsodetermine the tendency of the user to be “bad at” exercise in generaland not want to continue.

The health and beauty prediction section 334 predicts health and beautyinformation of the health and beauty environment information and healthand beauty feeling information of the personal preference evaluationinformation from a predicted menstrual cycle in the health and beautyinformation of the health and beauty environment information,meteorological forecast of the external service content informationperiodically acquired from the external service server apparatus 400,and action planning of the personal preference evaluation information.From the predicted health and beauty information of the health andbeauty environment information, another health and beauty informationmay also be predicted. FIG. 7 shows an example of the health and beautyprediction information, and health and beauty prediction processing canbe performed as described below.

1) From meteorological forecast of weather and atmospheric temperaturein the external service content information, a thermographic pattern ofthe health and beauty information and a health and beauty feeling of thepersonal preference evaluation information are predicted. Specifically,from a group of past time-series data of health management information,the health and beauty prediction section 334 can previously specifyhealth management information with a thermographic pattern of “highcoldness of lower body” and a joint pain of the health and beautyfeeling of “intense” and know that the weather of meteorologicalinformation was “rainy” and the atmospheric temperature was “10° C. orlower” at the time when that health management information was produced.Thus, for example when a meteorological forecast three days from nowshows that the weather is “rainy” and the atmospheric temperature is“10° C. or lower,” the health and beauty prediction section 334 canproduce health and beauty prediction information three days from nowincluding a thermographic pattern of “warning against coldness of lowerbody” and a joint pain of the health and beauty feeling of “intense.”

Similarly, when a meteorological forecast for the day after tomorrowshows that the weather is “cloudy” and the atmospheric temperature is“lower than 14° C.,” the health and beauty prediction section 334 canproduce health and beauty prediction information for the day aftertomorrow including a thermographic pattern indicating “caution againstcoldness of lower body” and a joint pain of the health and beautyfeeling of “medium.” When the above meteorological conditions are notmet, for example in a weather forecast for tomorrow, the health andbeauty prediction section 334 can produce health and beauty predictioninformation for tomorrow including a thermographic pattern indicating“warry-free” and a joint pain of the sense of health and beauty feelingof “none.”

2) Based on a meteorological forecast of dispersing pollen count in theexternal service content information, the health and beauty predictionsection 334 can produce health and beauty prediction informationincluding a sneeze prediction of the health and beauty information of“worry-free” when the dispersing pollen count is lower than 50/m³,“caution” when the dispersing pollen count ranges from 50/m³ to 250/m³,and “warning” when the dispersing pollen count is 250/m³ or higher.Similarly to the above case, from a group of past time-series data ofhealth management information, the health and beauty prediction section334 can find the relationship between the number of sneezes and the pastdispersing pollen count and set the dispersing pollen counts associatedwith “worry-free,” “caution,” and “warning” in advance.

3) The health and beauty information can be predicted from the actionplanning of the personal preference evaluation information to producehealth and beauty prediction information in which the degree of faceswelling is changed to “warning” and the level of breath odor is changedto “warning” for the day after a women's party in a schedule.

4) When the degree of face swelling of the health and beauty environmentinformation is “warning” and the skin moisturizing level is “caution” or“warning,” health and beauty prediction information can be produced witha skin condition score of “warning.” When the degree of face swelling is“caution” and the skin moisturizing level is “caution,” health andbeauty prediction information can be produced with the skin conditionscore changed to “caution.” As described above, health and beautyinformation on the day after a women's party can be found from a groupof past time-series data of health management information, so thathealth and beauty prediction information five days after the women'sparty can be produced as described above.

The recommendation section 335 combines the result of time-series dataanalysis in the time-series data analysis section 331, the result ofperformance effect evaluation of health and beauty actions in theperformance effect evaluation section 332, the result of user preferencecharacteristic evaluation in the user preference evaluation section 333,the result of prediction of health and beauty status and health andbeauty feeling in the health and beauty prediction section 334, and theresult of collective intelligence analysis in the collectiveintelligence data analysis section 337 to calculate information forrecommending selection, performance, and introduction timing of healthand beauty actions, health and beauty commodities, and health and beautyservices appropriate for each user, and records the recommendationinformation in the storage section 314.

1) When the recorded result of preference of a health and beauty actioncontent shows that the user is “bad at” the content in the health andbeauty action preference information, the recommendation section 335 canswitch that content to a different health and beauty service content ofthe same type. For example, when the user is bad at “exercise #1,” thisis switched to an easier content “exercise #2.” When the record showsthat the use is “bad at” a predetermined number or more of differentexercise contents, it can be determined that the user is “bad at”exercise in general, and the maximum number of reproduced contents perday can be set at “one at most.”

2) The recommendation section 335 searches collective intelligenceanalysis information about a thermographic pattern indicating “type ofcoldness of lower body” of users for a ranking of health and beautyactions with high performance rates in a group of users belonging to thepattern on each date or within a predetermined time period or a rankingof health and beauty actions with high counts for each personalpreference such as “pleasant”, “preference”, or “good at,” excludesactions which many of the users are “bad at” or did not perform in thehealth and beauty action preference, and health and beauty actions,commodities, services, or exercise types having small effects in theresult of performance effect evaluation, and selects a health and beautyaction determined to have a high performance effect as a health andbeauty content which is a candidate for deliver to the user at the timeof start of “follicular phase” in health and beauty prediction.

3) When the user has a living behavior pattern of “active in morning”and frequently performs health and beauty actions after getting up inthe morning, the recommendation section 335 can select a health andbeauty content relating to a health and beauty action, commodity,service, or health and beauty event desirable in the morning. From therelationship with performance locations (location information) of healthand beauty actions included in preference information of the user, therecommendation section 335 can select a health and beauty content whichcan be performed at a location preferred by the user.

4) When an update is performed in a health and beauty action, commodity,or service, or in a status of holding or participation of a health andbeauty event by a user serving as the user's model, the recommendationsection 335 can select the update as information which should betransmitted to the mirror apparatus 100 or the mobile communicationapparatus 500 accessed by the user first after the update.

The health and beauty contents may be limited depending on the type ofdevice owned by the user and recorded in the user terminal managementinformation in the storage section 314. For example, each health andbeauty content has a type of device stored therein which is recommendedfor reproduction. A content involving whole-body exercise or massage maybe selected as a recommendation to a user who has the device of the type“mirror apparatus (large)” and may not be selected for a user who doesnot have such a device. When a content regarding a health and beautyaction, commodity, health and beauty event is displayed in an area or ata point based on the location information of the mobile communicationapparatus 500, the recommendation section 335 may be configured not toselect that content for a user who has no registration of the mobilecommunication apparatus.

The personal learning and planning section 336 can use the informationfor recommending selection, performance, and introduction timing ofhealth and beauty actions, health and beauty commodities, and health andbeauty services appropriate for each user calculated in therecommendation section to produce a plan to deliver and reproduce ahealth and beauty service content for transmission to a predetermineduser and can record the plan in the storage section 314. FIG. 8 is adiagram showing an example of health and beauty planning information,and personal study planning processing can be performed as describedbelow.

1) When the personal preference evaluation information includes atoday's physical condition and a tomorrow's physical conditionprediction of “poor,” a health and beauty service of “Pilates class #1”and a health and beauty event of “trial yoga lesson #4” are set toreproduction or reproduction scheduled.

2) At the time when the menstrual cycle of health and beauty environmentinformation changes from “menstrual phase” to “follicular phase,” then“healing BGM #3” is recorded in health and beauty content planninginformation, and a schedule of healing BGM reproduction is set to“healing BGM #3” from “healing BGM #2.”

3) After the user takes a bath with a thermographic pattern predictionof health and beauty environment information of “caution againstcoldness of lower body” or “warning against coldness of lower body,” ahealth and beauty service of “bedrock bath #3” and a health and beautyevent of “moxibustion lesson #1” are set to reproduction scheduled.

4) After the user takes a bath on the day when an action planning ofpersonal preference evaluation information includes “dating,” a healthand beauty service of “nail salon #2,” and health and beauty commoditiesof “organic cosmetics #2” and “beauty device #2” are set to reproductionscheduled.

5) After the user takes a bath (3/26) three days before the day (3/29)when an action planning of personal preference evaluation informationincludes “women's party,” a health and beauty service of “women's travel#2” is set to reproduction scheduled.

The collective intelligence analysis information is information obtainedfrom analysis of overall health information of a plurality of users andis provided by extracting average movements or change patterns oftime-series data on each date or within a predetermined time period fromhealth and beauty information of health and beauty environmentinformation, histories of living behaviors, and health and beautyfeelings of personal preference information for each user groupsatisfying a predetermined condition.

For example, the collective intelligence data can be produced by using,as a key, meteorological information of external service contentinformation, environment information of health and beauty environmentinformation, and health and beauty action information. The predeterminedcondition is the sex, age, and date of time-series data, and they can beset as attributes of user groups. The attributes of the user groups maybe set on the basis of classification types, classification patterns, ornumerical values of data items included in health information.

The collective intelligence data analysis section 337 first extracts auser group satisfying the predetermined condition, and then uses thetime-series data of each health information of the extracted user groupto produce, as collective intelligence analysis information, numericalvalue information such as average, highest value, and lowest value ofhealth and beauty environment information, health and beauty actioninformation, and personal preference information for each day, andcharacteristic information such as average movements and change patternsof numerical values within a predetermined time period.

Specifically, 1) the collective intelligence data analysis section 337calculates numerical data such as average value, maximum value, minimumvalue, and variance value for each age of women and on each day in eachdata item of health and beauty information of health and beautyenvironment information or calculates an aggregated value of patterns ofthe menstrual cycle, thermographic pattern, and facial expression.

2) The collective intelligence data analysis section 337 can calculate apercentage or an average value for each time zone for each age of womenand on each date or for each age and within a predetermined time period,in each data item of the living activity record in personal preferenceinformation. For example, the collective intelligence data analysissection 337 can calculate an average value or a percentage in each timezone of get-up times for each age, or the number of alcohol intakeswithin a predetermined time period.

3) The collective intelligence data analysis section 337 can calculate aranking of health and beauty actions with high performance rates on eachdate or within a predetermined time period in each of user groupsclassified by health and beauty type such as “coldness of fingers andtoes” or “coldness of lower body,” or a ranking of health and beautyactions with high counts for each personal preference information suchas “pleasant,” “preference,” or “good at.”

The collective intelligence data analysis section 337 can producecollective intelligence data by using, as a key, meteorologicalinformation of external service content information, indoor environmentinformation of health and beauty environment information, or health andbeauty action of health and beauty action information. For example, thefollowing numerical values or characteristics can be produced.

1) The collective intelligence data analysis section 337 can calculatean average or an aggregated value of the same patterns of skinmoisturizing level for each age, skin condition score, thermographicpattern, and sleep quality in an indoor humidity of 5% increments inindoor environment information of health and beauty environmentinformation.

2) The collective intelligence data analysis section 337 can aggregatean average value of health and beauty information, and patterns ofmenstrual cycle, thermographic pattern, and facial expression calculatedfor each age of women in years, months, and weeks for each weather suchas “sunny,” “cloudy,” and “rainy.”

3) The collective intelligence data analysis section 337 can calculatean average value of performances or nonperformances of a health andbeauty action, maximum value and minimum value of numbers ofperformances, and average value, maximum value, and minimum value ofamounts of performance of a health and beauty action for each age ofwomen in years, months, and weeks for each health and beauty action ofhealth and beauty action information.

4) In the 3) described above, for each health and beauty action ofhealth and beauty action information, the collective intelligence dataanalysis section 337 can create an average of changes in numerical valueof health and beauty information of health and beauty environmentinformation, pattern of the changes in numerical value, and changes infacial expression and thermographic pattern in accordance with theperformance or nonperformance and the performance amount of a health andbeauty action for each age of women in years, months, and weeks.

FIGS. 9 to 11 are diagrams showing processing flows in the healthinformation service system according to the present embodiment.

Upon sensing the presence of a user in front of the mirror apparatus100, the human detection sensor 150 of the mirror apparatus 100transmits user presence sensing information to the control section 220of the control apparatus 200. After the transmission of the userpresence sensing information, the human detection sensor 150 senses thepresence or absence of the user at regular time intervals, and transmitsuser absence sensing information to the control section 220 when theuser is absent (ST1).

Upon reception of the user presence sensing information from the humandetection sensor 150, the control section 220 authenticates the identityof the user present in front of the mirror apparatus 100 (ST2). Theauthentication of the identity can be performed, for example, bybiometric authentication such as face image recognition, veinauthentication, iris authentication, and voiceprint authentication. Inaddition to the biometric authentication, authentication information maybe used such as a password which can be associated with a user IDidentifiable by the mirror apparatus 100 or the service server apparatus300. When the authentication result is OK, the control section 220transmits a user ID and a device ID of the mirror apparatus 100 as wellas the authentication result to the service server apparatus 300 (ST3).When the user and the mirror apparatus 100 have a one-to-onerelationship such as when a resident is alone in a living space, thehuman detection sensor 150 detects the presence of a user and then thecontrol section 200 can transmit the user ID of the user and the deviceID of the mirror apparatus 100 to the service server apparatus 300without performing the authentication of the identity of the user.

After the authentication of the identity of the user, the controlsection 220 outputs a measurement start signal to the sensor controlsection 231 in order to perform processing of measuring the health andbeauty status of the user with the health and beauty environment sensorsection, and controls the components constituting the health and beautyenvironment sensor section to acquire health and beauty environmentinformation (ST4). The acquired health and beauty environmentinformation is transmitted to the control section 220. The function ofacquiring the health and beauty environment information is describedabove.

The control section 220 stores the acquired health and beautyenvironment information in association with the user ID in the storagesection 213 (ST5). The control section 220 retrieves necessaryinformation linked to the user ID at this point in time, from userinformation, theme content information, local cache information, andprivacy content information in the storage section 213 (ST6). Then, thecontrol section 220 outputs the health and beauty environmentinformation, user information, theme content information, local cacheinformation, and privacy content information to the output controlsection 212. The output control section 212 can display the informationreceived from the control section 220 on the display screen 113 or canoutput the information in audio form from the speaker 140.

In transmitting the health and beauty environment information to theservice server apparatus 300 over the network, the control section 220can perform the privacy processing described above to transmit thehealth and beauty environment information which has been subjected tothe privacy processing (quantifying processing) to the service serverapparatus 300.

Next, the control section 220 transmits the health and beautyenvironment information in association with the user ED and the deviceID of the mirror apparatus 100 to the service server apparatus 300(ST7). The control section 320 of the service server apparatus 300receives the health and beauty environment information at thecommunication section 311 and stores the information in the storagesection 314. At this point, the service server apparatus 300 may performauthentication processing by using the user ID or the device ID.

The control section 320 can stores the received health and beautyenvironment information in a health and beauty environment informationDB of the storage section 314 by using the user ID and the device ID ofthe mirror apparatus 100 as a key (ST8). As shown in FIG. 6, the controlsection 320 stores the health and beauty environment informationregarding user's health acquired by the health and beauty environmentsensor section in time sequence as daily health and beauty environmentinformation of the user. For example, the information can be stored oneach date of the acquisition of the health and beauty environmentinformation, or can be stored on each date and at each acquisition timewhen the health and beauty environment information is acquired aplurality of times in a day.

The control section 320 of the service server apparatus 300 acquiresexternal service content information from the external service serverapparatus 400 through the external content acquisition section 312(ST9). The external content acquisition section 312 stores the externalservice content information acquired from the external service serverapparatus 400 in an external service content information DB of thehealth and beauty management information. Since the external servicecontent information is used as information common to users, it may bestored not for each user but in a separate database common to the users.

The processing of acquiring the external service content information maybe performed not in synchronization with the processing of storing thehealth and beauty environment information transmitted from the mirrorapparatus 100 (control apparatus 200) (a request to store the health andbeauty environment information from the mirror apparatus 100). Theexternal service content information such as meteorological informationor traffic information can be configured to be acquired at regular timeintervals in a day such that the external service content informationcan be updated to the latest version at any time.

The control section 320 stores, for example, meteorological informationon each date in the health and beauty management information stored intime sequence (ST10).

The control section 320 transmits the external service contentinformation acquired from the external service server apparatus 400 tothe mirror apparatus 100 (control apparatus 200) through thecommunication section 311. The control section 320 transmits theexternal service content information to the mirror apparatus 100 at thetime of acquisition of the external service content information or atthe time of delivery specified in advance (ST11).

Upon reception of the external service content information from theservice server apparatus 300, the control section 220 of the controlapparatus 200 stores the external service content information in thestorage section 213 as local cache information (ST12). When the humandetection sensor 150 detects that the user is present within the sensingrange of the mirror apparatus 100, the control section 220 can performcontrol to output the received external service content information tothe output control section 212 for display on the display screen 113 orfor audio output.

The control section 320 of the service server apparatus 300 performscontrol to extract the past health and beauty environment informationacquired over a certain time period necessary for time-series dataanalysis and the preceding time-series data analysis information foreach user ID to perform time-series data analysis processing. Thetime-series data analysis section 331 performs time-series data analysis(ST13) based on the latest health and beauty environment information,the extracted past health and beauty environment information, and thepreceding time-series data analysis information, and stores the analysisresult (for example, a data time-series movement pattern) in the storagesection 314.

Next, description is made of processing of acquiring a health and beautyaction of the user (ST14). FIG. 12 is a diagram showing a processingflow of acquiring an action status of the user regarding health andbeauty in the health information service system according to the presentembodiment.

As shown in FIG. 12, the control section 220 of the control apparatus200 performs control such that health and beauty environment informationis measured by the health and beauty environment sensor section (S101),and acquires the health and beauty environment information measuredbefore performance of a health and beauty action from the health andbeauty environment sensor section and stores the information inassociation with the user ID in the storage section 213 (S102).

The control section 220 acquires a predetermined health and beautyaction content stored in the storage section 213 and controls the outputcontrol section 212 to reproduce the predetermined health and beautyaction content (S103). With the reproduction of the health and beautyaction content, the control section 220 controls the user actionrecognition section 233 to start processing of recognizing the healthand beauty action being performed by the user (S104).

The user action recognition section 233 calculates location informationof each region such as user's face, arm and leg, and torso in images orvideos taken by the camera 120 or acquired by the motion sensor, matchesthe location information with the corresponding positions in displayeddata of the health and beauty action content displayed on the displayscreen 113, and makes a measurement to determine whether or not the useris performing the health and beauty action in synchronization with thehealth and beauty action content being reproduced. The control section220 displays the result of recognition of the health and beauty actioncontent and the user (for example, information about the health andbeauty action being performed, performance or nonperformance,performance start time, performance end time, number of performances,and performance score) together with the health and beauty actioncontent being reproduced on the display screen 113 (S105).

When the reproduction of the health and beauty action content iscompleted (YES at 106), the control section 220 performs control todisplay the result of recognition performed in the user actionrecognition section 233, for example, the progress or completion of thehealth and beauty action of the user on the display screen 113, andperforms processing of acquiring health and beauty environmentinformation with the health and beauty environment sensor section.

The control section 220 stores the result of recognition of the healthand beauty action and the health and beauty environment informationbefore and after the health and beauty action in association with theuser ID in the local cache information (S107). At this point, thecontrol section 220 can perform control to display the health and beautyenvironment information and changes in health and beauty status beforeand after the health and beauty action on the display screen 113.

The control section 220 transmits the information of the health andbeauty action performed by the user and the health and beautyenvironment information before and after the health and beauty action inassociation with the user ID and the device ID of the mirror apparatus100 to the service server apparatus 300 (S108). When the user continuesthe health and beauty action after the reproduction of the health andbeauty action content is completed, the control section 220 may beconfigured to continue the recognition processing in the user actionrecognition section 233 until the user finishes the health and beautyaction.

Upon reception of the user ID, the device ID, the health and beautyenvironment information before and after the health and beauty action,and the health and beauty action information at the communicationsection 311, the service server apparatus 300 stores them in associationwith the health and beauty environment information and the health andbeauty action information in the storage section 314 (ST15 in FIG. 10).

The control section 320 of the service server apparatus 300 performscontrol to perform processing of evaluating the performance effect of ahealth and beauty action. The performance effect evaluation section 332reads out health and beauty environment information on a predetermineddate or within a predetermined time period stored in a health and beautyaction information DB and health and beauty action information recordedin the health and beauty action information DB. The performance effectevaluation processing may be performed by taking account of themeteorological information recorded in the external service contentinformation DB or the living activity record recorded in a personalpreference evaluation DB.

For example, the performance effect evaluation section 332 extracts ahealth and beauty action performed by the user when a time-series changepattern is extracted by the time-series data analysis section 331 beforeand after the health and beauty action based on the action history, anduses the health and beauty environment information before and after thehealth and beauty action to produce performance effect evaluationinformation for the extracted health and beauty action (ST16). Asdescribed above, the performance effect evaluation section 332 mayextract a health and beauty action not performed by the user when atime-series pattern is extracted in a nonperformance period of thehealth and beauty action based on the action history and may use thehealth and beauty environment information in the nonperformance periodto produce non-performance effect evaluation information for theextracted health and beauty action. The time-series pattern includes atime-series pattern representing no change of the health and beautystatus of the user.

The control section 320 stores the produced performance effectevaluation information in the health and beauty action information DB ofthe storage section 314 (ST17) and transmits the user ID, the device ID,and the performance effect evaluation result of the health and beautyaction to the mirror apparatus 100 (control apparatus 200) (ST18).

The control section 220 of the control apparatus 200 stores the user IDand the performance effect evaluation result of the health and beautyaction received from the service server apparatus 300 in the local cacheinformation of the storage section 213 (ST19). The control section 220can perform control to display the received performance effectevaluation result of the health and beauty action on the display screen113, and the output control section 212 displays the performance effectof the health and beauty action on the display screen 113 to the userbased on an instruction from the control section 220.

Next, description is made of user preference acquisition processing anduser preference evaluation processing. The control section 220 of themirror apparatus 100 senses a human appearing in front of the mirrorapparatus 100 with the human detection sensor 150 and performsauthentication to identify a user ID, and then performs control toperform processing of acquiring user preference characteristicinformation with the user preference acquisition section 236 uponoperation by the user or at an active arbitrary time (ST20).

The user preference acquisition section 236 displays a predeterminedcontent for collecting user preference characteristics on the displayscreen 113 through the output control section 212. The user preferenceacquisition section 236 acquires the result of response of the user tothe user preference characteristics collecting content and stores it inthe local cache information of the storage section 213 (ST21).

The control section 220 transmits the user ID, the device ID, userpreference characteristic collecting information, and the result of theuser response to the user preference characteristic collecting contentto the service server apparatus 300 (ST22). The user preferencecharacteristic collecting information is information indicating whichuser preference characteristics are collected. The user preferencecharacteristic collecting content can be produced by the user preferenceevaluation section 333 of the service server apparatus 300, and caninclude a user preference characteristic collecting content ID foridentifying each content. Since the response result is transmitted tothe service server apparatus 300, it may be configured not to be storedas the local cache information in the storage section 213.

Upon reception of the user ID, the device ID of the mirror apparatus100, the user preference characteristic collecting information, and theresult of the user response to the user preference characteristiccollecting content at the communication section 311, the control section320 of the service server apparatus 300 stores them in the personalpreference evaluation DB of the storage section 314 (ST23). As describedabove, the user preference information may be acquired in the mobilecommunication apparatus 500, and the service server apparatus 300 maystore the result of response of the user to the user preferencecharacteristic collecting content received from the mobile communicationapparatus 500 in the personal preference evaluation DB of the storagesection 314.

The user preference evaluation section 333 analyzes each informationsuch as living activity record, health and beauty feeling, and healthand beauty action preference of each user stored in the personalpreference evaluation DB of the storage section 314 periodically or at apredetermined time, and stores the analysis result in association withthe user ID in the personal preference evaluation DB (ST24).

In addition, the health and beauty prediction section 334 of the serviceserver apparatus 300 can perform health and beauty prediction processingof predicting health and beauty information of the health and beautyenvironment information DB and health and beauty feeling information ofthe personal preference evaluation DB at a predetermined time from themenstrual cycle prediction of health and beauty information of thehealth and beauty environment information DE, the meteorologicalforecast of the external service content information DB periodicallyacquired from the external service server apparatus 400, and the actionplanning of the personal preference evaluation DB (ST25). As shown inFIG. 7, health and beauty prediction information obtained through thehealth and beauty prediction processing is stored in the storage section314, and the control section 320 transmits the produced health andbeauty prediction information to the mirror apparatus 100 (controlapparatus 200) at an arbitrary time. The control section 220 of thecontrol apparatus 200 stores the received health and beauty predictioninformation in the storage section 213 and performs control to displaythe health and beauty prediction information on the display screen 113in response to a user operation or at a predetermined time.

The recommendation section 335 of the service server apparatus combines,at a predetermined time, the result of time-series data analysis in thetime-series data analysis section 331, the result of performance effectevaluation of health and beauty actions in the performance effectevaluation section 332, the result of user preference characteristicsevaluation in the user preference evaluation section 333, the result ofprediction of health and beauty status and health and beauty feeling inthe health and beauty prediction section 334, the result of collectiveintelligence analysis in the collective intelligence data analysissection 337, and the information about the type of device held by theuser recorded in the user terminal management information to performrecommendation processing of producing recommendation informationindicating selection, performance/nonperformance, and introductiontiming of health and beauty actions, health and beauty commodities, andhealth and beauty services appropriate for each user (ST26), and recordsthe recommendation information in the storage section 314. The controlsection 320 transmits the produced recommendation information to themirror apparatus 100 (control apparatus 200) at an arbitrary time. Thecontrol section 220 of the control apparatus 200 stores the receivedrecommendation information in the storage section 213, and can displaythe recommendation information on the display screen 113 in response toa user operation or at a predetermined time or can add, update, orcorrect a health and beauty action content based on the recommendationinformation.

The personal learning and planning section 336 produces a plan todeliver and reproduce a health and beauty service content fortransmission to the user as shown in FIG. 8 based on the recommendationinformation produced in the recommendation section 335 (ST27) and storesthe plan in the health and beauty action information DB of the storagesection 314. The control section 320 transmits the produced health andbeauty planning information to the mirror apparatus 100 (controlapparatus 200) at an arbitrary time. The control section 220 of thecontrol apparatus 200 stores the received health and beauty planninginformation in the storage section 213 and can display therecommendation information on the display screen 123 in response to auser operation or at a predetermined time.

The collective intelligence data analysis section 337 performscollective intelligence analysis processing of grouping a plurality ofusers satisfying a predetermined condition at a predetermined time andanalyzing average movements or change patterns on each date or within apredetermined time period from health and beauty information of thehealth and beauty environment information DB, living activity record ofthe personal preference evaluation DB, and the health and beautytendency stored in the storage section 314 (ST28). The analysis resultfrom the collective intelligence analysis processing is stored in thestorage section 314 as numerical value information within a user groupsuch as average, highest value, and lowest value for each day or ascharacteristic information such as average movements or change patternsin a predetermined time period for each user group attribute(predetermined condition), similarly to the configuration of the healthand beauty environment information DB and the personal preferenceevaluation DB.

As described above, the health information service system according tothe present embodiment is configured to include the mirror apparatus 100including the mirror (for example, the half mirror 111) configured toreflect the mirror image of the user located in front of the mirror, thedisplay panel 112 provided on the rear side or front side of the mirrorand configured to display various types of information, and the imagingapparatus (camera 120) configured to take images of regions of the bodyof the user who is reflected in the mirror, the mirror apparatus 100being installed in a house where the user lives or a place which theuser visits, the measurement device (the camera 120, the sensor device160, the health and beauty measurement apparatus 170) configured tomeasure the health and beauty status of the user who uses the mirrorapparatus 100, the control apparatus 200 configured to control themeasurement device and the mirror apparatus 100, and the service serverapparatus 300 connected to the control apparatus 200 over the network.

The service server apparatus 300 associates health and beautymeasurement information acquired by the measurement device with thestatus of the action regarding health and beauty of the user imaged bythe imaging apparatus and stores the health and beauty measurementinformation and the action status daily in time sequence, and has thetime-series data analysis section configured to extract the time-seriespattern representing a change in the health and beauty status of theuser or the time-series pattern representing no change in the health andbeauty status of the user based on the time-series health and beautymeasurement information in the past, and the performance effectevaluation section configured to extract the health and beauty actionperformed by the user when the time-series pattern is found before andafter the health and beauty action of the user based on the history ofactions or the health and beauty action not performed by the user whenthe time-series pattern is found in the nonperformance period of thehealth and beauty action of the user based on the history of actions andto use the health and beauty measurement information before and afterthe health and beauty action or the health and beauty measurementinformation in the nonperformance period to produce performance ornonperformance effect evaluation information about the extracted healthand beauty action.

With such a configuration, the health and beauty statuses of the userand the statuses of the actions regarding health and beauty performed bythe user can be collected to assist in improvement of health and beautyof the user. Specifically, the mirror apparatus used daily by the userand acquiring the mirror image of the user can be used to collect thestatuses of the actions regarding health and beauty of the user, so thatthe actions regarding health and beauty associated with daily livingbehaviors and the associated information can be provided to assist inimprovement of health and beauty of the user.

In an example of assisting in improvement of health and beauty of theuser in this manner, a user A with “high coldness of lower body” usesthe health information service system according to the presentembodiment in the following flow.

1) The user A takes his/her thermographic images with the infraredcamera mounted on the mirror apparatus 100 before and after taking abath and checks the effect before and after the bath from changes inthermographic image, still image, and thermographic pattern for eachbody region. The thermographic pattern is transmitted to the serviceserver apparatus 300.

The time-series data analysis section 331 of the service serverapparatus 300 analysis time-series data representing changes inthermographic pattern before taking a bath acquired for the last monthand recorded in the health and beauty environment information DB, andoutputs (analyzes) the degree in which the health and beauty status hasbecome better as seen in the pattern of coldness of lower body changingfrom high to low. For example, coldness statuses including highcoldness, medium coldness, low coldness, and no coldness may be graphed.

3) The performance effect evaluation section 332 of the service serverapparatus 300 refers to meteorological information of the externalservice content information DB and a history of performances of “massageof foot trigger points #2,” “supplement #3,” and “exercise #1” set to beperformed by the user for the purpose of removing “coldness of lowerbody” recorded in the health and beauty action information DB andperforms the performance effect evaluation of those health and beautyactions. Although the atmospheric temperature was generally low and itwas often cloudy or rainy, it is determined that the “massage of foottrigger points #2” and “supplement #3” performed almost every day havethe effect in which the health and beauty status has become better asseen in 2) from the pattern of coldness of lower body changing from highto low, and it is determined that those health and beauty actions havethe effects for the user A at the present time.

4) For “exercise #1” hardly performed since a certain point in time, theuser preference evaluation section 333 of the service server apparatus300 acquires the fact that the user A is bad at and does not likeexercise in general through a user preference characteristic collectingcontent and records the fact in health and beauty action preferenceinformation of the personal preference evaluation DB.

5) The recommendation section 335 of the service server apparatus 300refers to a tendency in health and beauty changes and a ranking ofpopular health and beauty actions in a group of users with “highcoldness of lower body” of the same age and same residence area as theuser A calculated in the collective intelligence data analysis section337 to select a health and beauty content suitable for the user A. Inthis case, since the fact that the user A is bad at and does not likeexercise in general can be found in the personal preference evaluationinformation, the recommendation section 335 excludes the exercisecontents from the preference of the use A and selects a popularcommodity content of “ginger honey tea” as a new health and beautycontent based on a great interest and a high persistence rate infood-related contents.

6) The time-series data analysis section 331 of the service serverapparatus 300 can produce a change pattern of health and beauty statusof the user representing occurrence of “high coldness of lower body” ona day when the atmospheric temperature suddenly drops. The health andbeauty prediction section 334 can predict that “high coldness of lowerbody” is likely to occur (produce prediction health and beautyinformation) on the day when the meteorological forecast predicts asudden drop in atmospheric temperature. The personal learning andplanning section 336 records the fact that a commodity introductioncontent for “ginger honey tea” should be recorded on the mirrorapparatus 100 after the user takes a bath on the day when the suddendrop in atmospheric temperature is predicted.

7) When the recommendation section 335 of the service server apparatus300 determines that no schedule is present in the action planning of theuser A in the evening of the day when the atmospheric temperaturesuddenly drops based on the previously acquired living behavior planninginformation, the recommendation section 335 transmits the fact that a“moxibustion lesson” which the user is much interested in has a vacancyon that day and a participation guide as recommendation information tothe mobile communication apparatus 500 at the time when the user Aleaves the office based on location information of the mobilecommunication apparatus 500. Since the recommendation section 335 canfind that the user is interested in the “moxibustion lesson” in advancefrom the preference evaluation information of the user, therecommendation section 335 can provide the recommendation information inresponse to the fact that no schedule is present in the living actionplanning information and the location information of the mobilecommunication apparatus 500.

In another example of assisting in improvement of health and beauty ofthe user, the health information service system according to the presentembodiment is applicable to a hair salon as described above. Forexample, the past hair style of the user acquired by the camera 120through the mirror apparatus 100 can be stored in the control apparatus200 or the service server apparatus 300 and can be displayed on thedisplay screen 113 with a mirror image of the user side by side. Theuser can compare the past hair style with the present hair style todetermine the arrangement of his/her hair style under the advice of ahairdresser. It is possible to display images or videos of popular hairstyles or hair colors suitable for the shape of the user's face such asa round face, hair styles or hair colors recommended by the hairdresser,or hair styles or hair colors wanted by the user as well.

The camera 120 of a wearable type can acquire the drying or damagedegree of user's hair or the scalp condition as health and beautyinformation of the user (health and beauty environment information).Performance statuses of use of a conditioner for hair, scalp massage, orspecial treatment for head and hair including massage can be acquired ashealth and beauty actions of the user through the mirror apparatus 100.The control apparatus 200 can acquire the health and beauty actions ofthe user with control of an interface for acquiringperformance/non-performance of the use of the conditioner or the likethrough the display screen 113. Then, the performance effect evaluationfor the health and beauty actions can be performed as described above byacquiring health and beauty information such as the drying or damagedegree of user's hair or the scalp condition before and after the healthand beauty actions such as the use of the conditioner.

In the health information service system described above, therecommendation information produced by the recommendation section 335can be configured to include first recommendation information producedaccording to the location information of the mobile communicationapparatus 500 and second recommendation information associated with thefirst recommendation information, different from the firstrecommendation information, and provided for transmission to the mirrorapparatus 100. For example, as shown in FIG. 13A, the service serverapparatus 300 starts recommendation processing (ST26_1) and acquireslocation information from the mobile communication apparatus 500 of theuser (ST29, ST26_2). When the user is absent at the location where themirror apparatus 100 is installed (NO at ST26_3), for example, theservice server apparatus 300 can extract (retrieve) aesthetic saloninformation for face massage serving as a health and beauty actioncontent based on the location information as recommendation information(ST26_4) instead of a health and beauty action content representing facemassage to be provided for the mirror apparatus 100, and provides theinformation for the mobile communication apparatus 500 (ST26_5). Whenthe user is present at the location where the mirror apparatus 100 isinstalled (YES at ST26_3), the service server apparatus 300 can transmitthe recommendation information only to the mirror apparatus 100 or cantransmit the recommendation information to the mobile communicationapparatus 500 as well as the mirror apparatus 100. In this manner, therecommendation section 335 can be configured to select as appropriatethe recommendation information based on the user location informationacquired from the mobile communication apparatus 500 and transmit theselected information to the mobile communication apparatus 500 or themirror apparatus 100 (ST26_6, ST26_7). The location information of themirror apparatus 100 can be stored (acquired) in advance as userterminal management information shown in FIG. 5 in association with thedevice ID, for example.

The recommendation section 335 can provide different recommendationinformation pieces for the mobile communication apparatus 500 based onuser location information acquired from the mobile communicationapparatus 500. In this case, each of the recommendation informationpieces can be stored in association with different location informationof the mobile communication apparatus 500. As shown in FIG. 13B, therecommendation section 335 starts recommendation processing (ST26_8),and first acquires location information from the mobile communicationapparatus 500 of the user present at a position A (ST29_1, ST26_9).Based on the position A of the user acquired from the mobilecommunication apparatus 500, the recommendation section 335 can extractrecommendation information A associated with the corresponding locationinformation from the plurality of recommendation information pieces andtransmit the recommendation information A to the mobile communicationapparatus 500 (ST26_10, ST26_11). The recommendation section 335 alsoacquires location information from the mobile communication apparatus500 of the user present at a position B different from the position A(ST29_2, ST26_12). Based on the position B of the user acquired from themobile communication apparatus 500, the recommendation section 335extracts recommendation information B associated with the correspondinglocation information from the plurality of recommendation informationpieces (ST26_13). The recommendation information B is different from therecommendation information A. The recommendation section 335 cantransmit the recommendation information B to the mobile communicationapparatus 500 of the user present at the position B (ST26_14). Forexample, in accordance with the user location information, informationabout an aesthetic salon near the location where the user is present canbe provided as recommendation information for the mobile communicationapparatus 500. In addition, aesthetic salon information or gyminformation can be provided in accordance with the location information.

As described above, the mobile communication apparatus 500 of the usercan have the measurement function of measuring health and beautystatuses of the user and the action status acquirement function ofacquiring a history of actions regarding health and beauty performed bythe user, similarly to the mirror apparatus 100 and the controlapparatus 200. Thus, the health information service system according tothe present embodiment can be configured to acquire the measurementinformation representing health and beauty status measurements and thehistory of actions of the user from both the mirror apparatus 100 andthe mobile communication apparatus 500, and the service server apparatus300 can provide the health information service described above asappropriate for one or both of the mirror apparatus 100 and the mobilecommunication apparatus 500.

The mobile communication apparatus 500 of the user can acquire locationinformation in acquiring health and beauty environment information or inacquiring a history of actions regarding health and beauty performed bythe user. The mobile communication apparatus 500 can transmit theacquired health and beauty environment information, health and beautyaction information, location information, user ID, and device ID inassociation with each other to the service server apparatus 300. Theservice server apparatus 300 can stores the health and beautyenvironment information and the health and beauty action informationassociated with the user ID, the device ID, and the location informationin the storage section 314. The service server apparatus 300 can performa time series analysis of correlation between the health and beautyaction information and the location information representing theperformance location of the action to calculate a personal tendency andcan provide the user with a recommendation content for each locationinformation suitable for the personal tendency.

For example, as shown in FIG. 14, upon acquisition of health and beautyaction information (ST31), the mobile communication apparatus 500acquires location information associated therewith. The mobilecommunication apparatus 500 transmits the user ID, the device ID of themobile communication apparatus 500, the health and beauty actioninformation, and the location information to the service serverapparatus 30 (ST32). The service server apparatus 300 stores the healthand beauty action information and the location information representingthe performance location of the health and beauty action in the storagesection 314 in association with the user ID and the device ID (ST33).

The service server apparatus 300 performs performance effect evaluationfor the location information of the health and beauty action (ST34) andstores performance effect evaluation result information in the storagesection 314 (ST35). The mobile communication apparatus 500 acquires userpreference about the health and beauty action (ST36) and then transmitsthe acquired user preference information to the service server apparatus300 (ST37). The service server apparatus 300 stores the acquired userpreference information in the storage section 314 (ST38). The serviceserver apparatus 300 evaluates the user preference of that user from theuser preference information, health and beauty action, and locationinformation to evaluate a personal preference tendency including thelocation information, user preference, and health and beauty action(ST39). For example, from location information recorded together withthe performance of a health and beauty action, the service serverapparatus 300 can find the location where the user has performed thehealth and beauty action, and from preference information in performinga health and beauty action, the service server apparatus 300 can findthe location where the user likes to perform the action.

The service server apparatus 300 selects a health and beauty contentrecommended to the user in light of the location information based onthe performance effect evaluation information of the health and beautyaction for the location information and the personal preference tendency(ST40). For example, from the current location information of the useracquired from the mobile communication apparatus 500, the service serverapparatus 300 can find the location of performance of a health andbeauty action in the past and the health and beauty action performed atthat location by choice. Thus, based on the current position of the userand the location information associated with the history of actions, therecommendation section 335 can extract, as recommendation information, ahealth and beauty action which can be performed by the user at thecurrent position and transmit it to the mobile communication apparatus500. The service server apparatus 300 can produce a personal study planto allow reproduction of the health and beauty content in predeterminedlocation information suitable for the user preference and record theplan together with predetermined location information in a health andbeauty content planning DB of the storage section 314 (ST41).

Since the mobile communication apparatus 500 of the user can have themeasurement function of measuring health and beauty statuses of the userand the action status acquirement function of acquiring a history ofactions regarding health and beauty performed by the user, similarly tothe mirror apparatus 100 and the control apparatus 200, the healthinformation service system according to the present embodiment can beformed of the mobile communication apparatus 500 and the service serverapparatus 300. Thus, in the processing flow of the health informationservice system according to the present embodiment shown from FIG. 9 toFIG. 11, the mobile communication apparatus 500 can perform processingsimilar to that performed by the mirror apparatus 100 (control apparatus200) independently of or in parallel with the mirror apparatus 100(control apparatus 200).

Although the mirror apparatus 100 and the control apparatus 200 aredescribed as separate entities, they may be integrated. One controlapparatus 200 may be provided for a plurality of mirror apparatuses 100.In this case, each of the mirror apparatuses has a device ID and thecontrol apparatus 200 identifies the mirror apparatus 100 connectedthereto or communicating wirelessly therewith by the device ID. Thehealth and beauty measurement apparatus 170 may also be integrated withthe mirror apparatus 100. In addition, the mirror apparatus 100 and theservice server apparatus 300 may be integrated.

The processing in each of the control apparatus 200, the mobilecommunication apparatus 500, and the service server apparatus 300forming the health and beauty assisting system according to the presentembodiment may be realized by a program executable in a computer, andthe computer having the program installed thereon may operate as aninformation processing apparatus which performs each processing in thesystem according to the present embodiment. For example, the program maybe stored in an auxiliary storage apparatus, not shown, and a controlsection such as a CPU may read out the program stored in the auxiliarystorage apparatus to a main storage apparatus, the control section mayexecute the program read out to the main storage apparatus to cause thecomputer to perform each processing according to the embodiment.

The program can be recorded on a computer readable recording medium andprovided for a computer or downloaded to the computer over a networksuch as the Internet. The computer readable recording medium includes anoptical disk such as a CD-ROM, a phase change optical disk such as aDVD-ROM, a magneto-optical disk such as a Magneto-Optical (MO) disk andMD (Mini-Disk), a magnetic disk such as a Floppy® disk and removablehard disk, and a memory card such as a Compact Flash®, SmartMedia, SDmemory card, and memory stick. A hardware apparatus such as anintegrated circuit (including IC chip) designed and configuredparticularly for the purpose of the present invention is included in therecording medium.

Although the preferred embodiment of the present invention has beendescribed, that embodiment is illustrative and is not intended to limitthe scope of the present invention. The novel embodiment can beimplemented in various other forms, and various omissions,substitutions, and modifications can be made thereto without departingfrom the spirit or scope of the present invention. The embodiment andits variations are encompassed within the spirit or scope of the presentinvention and within the invention set forth in the claims and theequivalents thereof.

DESCRIPTION OF THE REFERENCE NUMERALS

-   100 MIRROR APPARATUS-   111 HALF MIRROR-   112 DISPLAY PANEL-   113 DISPLAY SCREEN-   120 CAMERA-   130 MICROPHONE-   140 SPEAKER-   150 HUMAN DETECTION SENSOR-   160 SENSOR DEVICE-   170 HEALTH AND BEAUTY MEASUREMENT APPARATUS-   200 CONTROL APPARATUS-   211 COMMUNICATION SECTION-   212 OUTPUT CONTROL SECTION-   213 STORAGE SECTION-   220 CONTROL SECTION-   231 SENSOR CONTROL SECTION-   232 MEASUREMENT DATA ANALYSIS SECTION-   233 USER ACTION RECOGNITION SECTION-   234 USER OPERATION RECOGNITION SECTION-   235 AR CONTROL SECTION-   236 USER PREFERENCE ACQUISITION SECTION-   300 SERVICE SERVER APPARATUS-   311 COMMUNICATION SECTION-   312 EXTERNAL CONTENT ACQUISITION SECTION-   313 OUTPUT CONTROL SECTION-   314 STORAGE SECTION-   320 CONTROL SECTION-   331 TIME-SERIES DATA ANALYSIS SECTION-   332 PERFORMANCE EFFECT EVALUATION SECTION-   333 USER PREFERENCE EVALUATION SECTION-   334 HEALTH AND BEAUTY PREDICTION SECTION-   335 RECOMMENDATION SECTION-   336 PERSONAL LEARNING AND PLANNING SECTION-   337 COLLECTIVE INTELLIGENCE DATA ANALYSIS SECTION-   400 EXTERNAL SERVICE SERVER APPARATUS-   500 MOBILE COMMUNICATION APPARATUS-   510 DISPLAY SECTION-   521 CAMERA-   522 SENSOR DEVICE-   523 MICROPHONE-   524 SPEAKER-   530 CONTROL SECTION-   531 COMMUNICATION SECTION-   532 OUTPUT CONTROL SECTION-   533 LOCATION INFORMATION ACQUISITION SECTION-   534 STORAGE SECTION-   535 SENSOR CONTROL SECTION-   536 MEASUREMENT DATA ANALYSIS SECTION-   537 USER ACTION RECOGNITION SECTION-   538 USER PREFERENCE ACQUISITION SECTION

The invention claimed is:
 1. A health information service systemcomprising: a measurement apparatus configured to measure a health andbeauty status of a user, a mirror apparatus having a first processorconfigured to acquire a history of actions regarding health and beautyperformed by the user, and a server connected over a network to themeasurement apparatus and the first processor, wherein the serverincludes: a storage device configured to associate health and beautymeasurement information about health and beauty of the user acquired bythe measurement apparatus with the history of actions and to store thehealth and beauty measurement information and the history of actions ofthe user daily in time sequence; and a second processor configured toextract a time-series pattern representing a change of the health andbeauty status of the user or a time-series pattern representing nochange of the health and beauty status of the user based on thetime-series health and beauty measurement information in the past andextract the health and beauty action performed by the user when thetime-series pattern is found before and after the health and beautyaction of the user based on the history of actions or the health andbeauty action not performed by the user when the time-series pattern isfound in a nonperformance period of the health and beauty action of theuser based on the history of actions and to use the health and beautymeasurement information before and after the health and beauty action orthe health and beauty measurement information in the nonperformanceperiod to produce performance or nonperformance effect evaluationinformation about the extracted health and beauty action, wherein themirror apparatus further includes a mirror configured to reflect amirror image of the user located in front of the mirror, a display panelprovided on a front side or a rear side of the mirror and configured todisplay various types of information, and an imaging apparatusconfigured to take an image of each region of a body of the user who isreflected in the mirror, the mirror apparatus being installed in a housewhere the user lives or a place which the user visits, and wherein thefirst processor is configured to show display data for guiding thehealth and beauty action on the display panel, to image an action statusof the user moving to match the display data with the imaging apparatus,and to transmit an action history of the user imaged on the basis of thedisplay data to the server.
 2. The health information service systemaccording to claim 1, wherein the mirror apparatus is formed of a halfmirror configured to reflect the mirror image of the user located infront and a display panel provided on a rear side of the half mirror andconfigured to output predetermined display light to be transmittedthrough the half mirror.
 3. The health information service systemaccording to claim 1, wherein the mirror apparatus is formed of a mirrorconfigured to reflect the mirror image of the user located in front anda display panel provided on a front side of the mirror and formed of atransparent display.
 4. The health information service system accordingto any one of claims 1 to 3, wherein: the measurement apparatus includesthe imaging apparatus as a measurement device configured to measure thehealth and beauty status of the user, and the imaging apparatus images avisible image, a near-infrared image, or a thermographic image, or videoof the user, and the first processor is configured to perform control toquantify data of the image or the video based on quantifying informationpreset for the image or video to produce the health and beautymeasurement information and to transmit the quantified health and beautymeasurement information to the server over the network, and not totransmit the image data before the quantification to the server.
 5. Thehealth information service system according to claim 1, wherein: thestorage device is configured to store the health and beauty measurementinformation and the history of actions daily, and meteorologicalinformation or a living activity record of the user in association witheach other, and the second processor configured to extract thetime-series pattern and to extract the meteorological information or theliving activity record related to the extracted time-series pattern as afactor in the change of the health and beauty status, the secondprocessor is further configured to produce, based on forecastinformation on the meteorological information or living behaviorplanning information corresponding to the living activity recordacquired in advance, prediction health and beauty measurementinformation of the user from the past health and beauty measurementinformation associated with the forecast information or the past healthand beauty measurement information associated with the living behaviorplanning information.
 6. The health information service system accordingto claim 1, wherein the second processor is further configured toproduce predetermined recommendation information based on the health andbeauty measurement information, the time-series pattern, the history ofactions or preference information of the user acquired in advance. 7.The health information service system according to claim 5, wherein thesecond processor is further configured to produce predeterminedrecommendation information based on the health and beauty measurementinformation, the time-series pattern, the history of actions orpreference information of the user acquired in advance, and theprediction health and beauty measurement information.
 8. The healthinformation service system according to claim 1, wherein the secondprocessor is further configured to produce predetermined recommendationinformation based on the health and beauty measurement information, thetime-series pattern, the history of actions or preference information ofthe user acquired in advance, and to transmit the recommendationinformation to the mirror apparatus for display on the display panel,wherein the recommendation information is configured to include firstrecommendation information produced according to location information ofa mobile communication apparatus held by the user connectable to theserver over a network and second recommendation information associatedwith the first recommendation information, different from the firstrecommendation information, and provided for transmission to the mirrorapparatus, and the second processor is configured to select the firstrecommendation information or the second recommendation informationbased on location information of the user acquired from the mobilecommunication apparatus and to transmit the selected recommendationinformation to the mobile communication apparatus or the mirrorapparatus.
 9. The health information service system according to claim1, wherein: the storage device is configured to store the health andbeauty measurement information and the history of actions daily, andmeteorological information or a living activity record of the user inassociation with each other, the second processor is configured toextract the time-series pattern and to extract the meteorologicalinformation or the living activity record related to the extractedtime-series pattern as a factor in the change of the health and beautystatus, the second processor further configured to: produce, based onforecast information on the meteorological information or livingbehavior planning information corresponding to the living activityrecord acquired in advance, prediction health and beauty measurementinformation of the user from the past health and beauty measurementinformation associated with the forecast information or the past healthand beauty measurement information associated with the living behaviorplanning information; and produce predetermined recommendationinformation based on the health and beauty measurement information, thetime-series pattern, the history of actions or preference information ofthe user acquired in advance, the prediction health and beautymeasurement information, and to transmit the recommendation informationto the mirror apparatus for display on the display panel, wherein therecommendation information is configured to include first recommendationinformation produced according to location information of a mobilecommunication apparatus held by the user connectable to the server overa network and second recommendation information associated with thefirst recommendation information, different from the firstrecommendation information, and provided for transmission to the mirrorapparatus, and the second processor is configured to select the firstrecommendation information or the second recommendation informationbased on location information of the user acquired from the mobilecommunication apparatus and to transmit the selected recommendationinformation to the mobile communication apparatus or the mirrorapparatus.
 10. A health information service system comprising: a mobilecommunication apparatus held by a user includes a first processorconfigured to: measure a health and beauty status of the user, andacquire a history of actions regarding health and beauty performed bythe user; and a server connected over a network to the mobilecommunication apparatus, wherein the server includes: a storage deviceconfigured to associate health and beauty measurement information abouthealth and beauty of the user acquired by the mobile communicationapparatus with the history of actions and to store the health and beautymeasurement information and the history of actions of the user daily intime sequence; and a second processor configured to extract atime-series pattern representing a change of the health and beautystatus of the user or a time-series pattern representing no change ofthe health and beauty status of the user based on the time-series healthand beauty measurement information in the past, extract the health andbeauty action performed by the user when the time-series pattern isfound before and after the health and beauty action of the user based onthe history of actions and to use the health and beauty measurementinformation before and after the health and beauty action to produceperformance effect evaluation information about the extracted health andbeauty action, and produce predetermined recommendation informationbased on the health and beauty measurement information, the time-seriespattern, the history of actions, the performance effect evaluationinformation, or preference information of the user acquired in advance,the recommendation information comprises a plurality of recommendationinformation pieces, each of the recommendation information pieces beingassociated with different location information of the mobilecommunication apparatus, and the second processor is configured toextract, based on location information of the user acquired from themobile communication apparatus, recommendation information associatedwith the location information from the plurality of recommendationinformation pieces and to transmit the extracted recommendationinformation to the mobile communication apparatus.
 11. A healthinformation service system comprising: a mobile communication apparatusheld by a user includes a first processor configured to: measure ahealth and beauty status of the user, acquire a history of actionsregarding health and beauty performed by the user, and acquire locationinformation, and a server connected over a network to the mobilecommunication apparatus, wherein the server includes: a storage deviceconfigured to associate health and beauty measurement information abouthealth and beauty of the user acquired by the mobile communicationapparatus with the history of actions, to store the health and beautymeasurement information and the history of actions of the user daily intime sequence, and to store location information associated with thehistory of actions, the location information indicating a position ofperformance of a health and beauty action of the user based on thehistory of actions; a second processor configured to extract atime-series pattern representing a change of the health and beautystatus of the user or a time-series pattern representing no change ofthe health and beauty status of the user based on the time-series healthand beauty measurement information in the past, extract the health andbeauty action performed by the user when the time-series pattern isfound before and after the health and beauty action of the user based onthe history of actions and to use the health and beauty measurementinformation before and after the health and beauty action to produceperformance effect evaluation information about the extracted health andbeauty action, and produce predetermined recommendation informationbased on the health and beauty measurement information, the time-seriespattern, the history of actions, the performance effect evaluationinformation, preference information of the user acquired in advance, orthe location information associated with the history of actions, and thesecond processor is configured to use current location information ofthe user acquired from the mobile communication apparatus and thelocation information associated with the history of actions, to extract,as the recommendation information, the health and beauty action whichcan be performed at a current position of the user based on the historyof actions, and to transmit the extracted recommendation information tothe mobile communication apparatus.
 12. A health information servicesystem comprising: a mirror apparatus including a mirror configured toreflect a mirror image of a user located in front of the mirror, and adisplay panel provided on a front side or a rear side of the mirror andconfigured to display health and beauty measurement informationregarding health and beauty of the user acquired by a measurementapparatus; a storage device configured to store the health and beautymeasurement information and a history of actions regarding health andbeauty performed by the user in connection with the acquisition of thehealth and beauty measurement information in association with eachother; and a processor configured to extract the health and beautymeasurement information in a nonperformance period of a health andbeauty action based on the history of actions from the storage deviceand to display the extracted information on the display panel.
 13. Ahealth information service system comprising: a measurement apparatusconfigured to measure a health and beauty status of a user, a firstprocessor configured to acquire a history of actions regarding healthand beauty performed by the user, and a server connected over a networkto the measurement apparatus and the first processor, wherein the serverincludes: a storage device-configured to associate health and beautymeasurement information about health and beauty of the user acquired bythe measurement apparatus with the history of actions and to store thehealth and beauty measurement information and the history of actions ofthe user daily in time sequence; and a second processor configured toextract a time-series pattern representing a change of the health andbeauty status of the user or a time-series pattern representing nochange of the health and beauty status of the user based on thetime-series health and beauty measurement information in the past andextract a first health and beauty action not performed by the user whenthe time-series pattern is found in a nonperformance period of a healthand beauty action of the user based on the history of actions and to usethe health and beauty measurement information in the nonperformanceperiod to produce nonperformance effect evaluation information about thefirst health and beauty action.